chenweize1998/fully-hyperbolic-nn

Code for paper Fully Hyperbolic Neural Networks

fully-hyperbolic-nn's Issues

Midpoint Definition

Hey,

Just wanted to ask, the paper defines the mid point as:

$\sqrt{1/k}\frac{\sum{v.x}}{\left|v.x \right|_L}$

but the code in kg/manifolds/lorentz :
$\sqrt{k}\frac{\sum{v.x}}{\left|v.x \right|_L}$
where: k=-K

is there any particular reason for this?

Unable to reproduce results

Hi @chenweize1998 - Thanks for doing great work. I was trying to reproduce results for machine translation but for some reason results seem to be way off. It'll be great if you can help me point in the right direction wrt what I might be doing wrong.

1. I am using Python 3.7.10, Pytorch 1.7.0+cu110, geoopt 0.3.1
2. When I train for iwslt14 as per instructions given in README here, the model gets stuck at around validation accuracy ~6 and validation perplexity ~880. After 40k steps, I see BLEU score of 0.

Let me know if I should add more details which might help you. I didn't make any changes to the code though.

about Definition 1

Hello Weize,

I have a mathematical problem when learning your great paper. Could you help me understand why Bx is on a hyperboloid. I tried my best to prove it, but failed. Thank you very much!

Best

ROC on airport lp dataset

Hi,

Thank you for your great work. I tried to reproduce the results on the airport lp dataset and I get 96.28 on the test dataset not 97.3 as reported in the paper. I used the same configuration parameters as you mentioned here in this repository.

NameError: name 'AverageAttention' is not defined

Hi,

When I'm trying to run the evaluation step using the command
bash eval_iwslt.sh 4 0 ~/fully-hyperbolic-nn-main/mt/model/iwslt/64/bs=10240_ac=1_dp=0.0_attdp=0.1_gn=0.5_ws=6000_lr=5/model_step_40000.pt
This error message comes up:
NameError: name 'AverageAttention' is not defined
I'm wondering where AverageAttention was first defined and imported. Thank you so much!

squared Lorentzian distance in code and paper

May I ask why squared Lorentzian distance is -2/k-2<a,b> in the code instead of 2/k-2<a,b> in the paper

Lorentz Linear result outside the manifold

I've been trying to implement a 2d convolution like operation using this method, I get the data and project it to the manifold, do a few steps and then pass them through the lorentz layer.
Just before I pass the data in, I perform the "check_on_manifold" function and it returns True. However, right after the layer it becomes False. Is there any specific reason for that? Should I reproject after using the layer?