Build classifier on top of BERT

[separius]

Build classifier on top of BERT

[Separius]

Hi @iliaschalkidis,
you can use train_model with from transformer.train import train_model and define a non token-level Task and train the model on your sentence-level task(which uses [CLS] internally)
you can take a look at tutorial.ipynb and read this in code.

[iliaschalkidis]

Hi @Separius, this looks really promising. Although I want to build something more complicated (custom), so I am going to read your code for train_model and start prototyping my model based on these principles.

I wish Google had already port BERT into tensorhub and made it as easy to use as ELMo tensorhub module.

Thanks a lot for your quick response! Have a great week!