Are Haste weights compatible with default PyTorch LSTM weights?

[lmnt-com]

Are Haste weights compatible with default PyTorch LSTM weights?

[sharvil]

In general, Haste and PyTorch weights aren't compatible because PyTorch doesn't support all of the features that Haste does (e.g. Zoneout, DropConnect, LayerNormLSTM). In the specific case where those features aren't used, the weights are sort of compatible – they're convertible from one format to the other.

Until we have full CPU support for Haste, we'll add methods to export weights so that PyTorch's native classes can use them if you're not using any of the Haste-specific features. I'll take this as a "please add CPU support to Haste" feature request.

[sharvil]

Added a couple of methods to haste.LSTM and haste.GRU so you can import/export weights into PyTorch native classes.

Example of converting Haste LSTM parameters to native PyTorch LSTM parameters:

haste_lstm = haste.LSTM(...)
native_lstm = nn.LSTM(...)

native_lstm.weight_ih_l0, \
    native_lstm.weight_hh_l0, \
    native_lstm.bias_ih_l0, \
    native_lstm.bias_hh_l0 = haste_lstm.to_native_weights()
native_lstm.flatten_parameters()

Example of converting native PyTorch parameters to Haste LSTM parameters:

native_lstm = nn.LSTM(...)
haste_lstm = haste.LSTM(...)

haste_lstm.from_native_weights(
    native_lstm.weight_ih_l0,
    native_lstm.weight_hh_l0,
    native_lstm.bias_ih_l0,
    native_lstm.bias_hh_l0)

Similar code applies to the GRU layer.

[sharvil]

Closing this out since we now have support for converting to/from PyTorch native weights. Haste layers also now run on CPU.