What are the major changes that were made?

[zeeshansayyed]

What are the major changes that were made?

[mukhal]

@zeeshansayyed Thank you for your interest to contribute. what I did here is simply use the masked language modeling architectures (encoders) and tweaked them a bit for sequence tagging in the sequence_tagger module.

I believe you can start with a vanilla BiLSTM (CRF would probably be more work, but still possible), which would be a significant contribution to the repo. The way I see it is that you'll need to implement LSTM encoder-only architecture with a similar interface to SequenceTagger. You can create something like LSTMSequenceTagger with a fairseq.models.LSTMEncoder inside.

Good luck!

[zeeshansayyed]

Hi @Mohammadkhalifa I opened an issue in the original fairseq repo asking about implementing the CRF tagger here. Do you have any thoughts on it? What I am mostly concerned about is having different implementations of forward while training and evaluation. If we do this, how will we have to change the generator code?
Thanks