cmd23333 / bert-tensorflow2.x Goto Github PK
View Code? Open in Web Editor NEWA tensorflow 2.x BERT implementation using League of Legends myth data(Chinese).
A tensorflow 2.x BERT implementation using League of Legends myth data(Chinese).
老哥,我用你的代码训练出来的ckpt模型,拿到bert4keras上面跑,结果报了这个错
Key bert/embeddings/word_embeddings not found in checkpoint
请问是什么问题呢
First of all, your code is clean and functional! I am glad that after reading your code, the reader will definitely learn a lot. I like it, so may I share some my suggestion?
1, your mlm task has only 11% acc, I think it is because your learning rate is too high, you should try 2e-5 as the google officially suggested
2, the mlm_predict=tf.matmul(x,self.embedding.token_embedding.embeddings, transpose_b=True) code may caused some warning of "variable is not traceable". the correct code should be implemented by senting a sequence of 0 to vocab_size to the embedding layer, the output is exactly the self.embedding.token_embedding.embeddings
after doing that i think you will get 99% of mlm task accuracy as me( i got 95+% acc just after 2 epoch), and the warning will go off~~
BTW, did you graduat from tju? the same to me, Good Luck to you my friend!
请问训练完成后的 token_embedding 要怎么读取?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.