Comments (2)
yl4579
commented on June 28, 2024
Not sure what you mean by other sorts of BERT. Maybe you can refer to the introduction section of my paper and MP-BERT paper?
As for the code, I will try to release the training and data preparation code and the pre-trained model by this week.
from pl-bert.
WhiteFu
commented on June 28, 2024
I am looking forward to it and thank you for your sharing:)
from pl-bert.
Related Issues (20)
- numbers range not converted correctly HOT 3
- Token indices sequence length is longer than the specified maximum sequence length HOT 3
- preprocess error HOT 2
- 2 Questions HOT 1
- RuntimeError: CUDA error: device-side assert triggered HOT 2
- the comparison between grapheme whole word classification HOT 1
- Modify the dataloader of Chinese PL-Bret HOT 2
- Possible Bug in Code HOT 1
- Proper format for dataset HOT 1
- Possible bug in masked index generation? HOT 9
- Possible bug in train.ipynb - num_vocab HOT 1
- Question regarding train.ipynb - num_tokens HOT 4
- RuntimeError: CUDA error: device-side assert triggered on criterion
- TypeError: unhashable type: 'list' HOT 3
- Irregular Loss Pattern ; getting "Loss:NaN" HOT 1
- No shards being saved HOT 3
- Vocab Loss = 0.0
- PL-Bert training for New Languages
- What are steps required to train the PL-bert model for new languages (specifically Hindi)
- How to preprocess a large text dataset (approximately 80 GB)
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pl-bert.