Could you please give me an API that tells me how could I create a RHN?

[jzilly]

Could you please give me an API that tells me how could I create a RHN?

[jzilly]

Thanks for being interested in RHNs. What exactly was the problem you encountered in your experiments?

What you may want to do is repurpose some of the code in: https://github.com/julian121266/RecurrentHighwayNetworks/blob/master/torch_rhn_ptb.lua
This file has a local function "rhn" which you could use. To ensure proper regularization, this function also needs differently initialized dropout masks as suggested by Gal https://github.com/yaringal/BayesianRNN. These dropout masks are also created in the provided file.

For now we have not written RHNs as a "module", so I am not sure whether it would work right away with the "Sequential" command. You should nevertheless be able to reuse parts of the code to address your problem. Of course we would be more than happy if you would like to implement an RHN module. For now we have tried to keep the code as close as possible to Gal's code (linked above).

[hfxunlp]

@julian121266 Thank you for your help, I want to find a model better than lstm and gru, and try it on some tasks like sequence classification and neural machine translation. I hope I could implement this as a module based on your code.