Comments (3)
armancohan
commented on August 23, 2024
This is expected as the Huggingface model was re-trained from scratch. However, the results are very similar.
If curious about the numbers, please see the README for the expected huggingface results and compare them with original results reported on our paper.
from specter.
malteos
commented on August 23, 2024
Thanks for the clarification. I missed that part in the README.
With "re-trained from scratch" you refer to training from the SciBERT checkpoint, right? And did you use the same paper triplets for training?
from specter.
armancohan
commented on August 23, 2024
Sure.
Yes, to clarify, by "from scratch" I mean starting from scibert checkpoint and continue training using the triplet loss objective.
from specter.
Related Issues (20)
- Nan loss in training specter with specter/scripts/pytorch_lightning_training_script/train.py
- ArrayField.empty_field problem with DatasetReader HOT 6
- json.decoder.JSONDecodeError: Expecting property name enclosed in double quotes: line 1 column 2 (char 1) HOT 1
- Training Data Set for Spectre HOT 5
- Error when training: "None" is not a <Class 'allennlp.data.fields.field.Field'> HOT 1
- How to change the sequence length? HOT 1
- How to deal with this bug?
- How to train my own model with other checkpoints?
- Use custom dataset without hard negatives
- Code does not go to completion when `njobs` is greater than 1 HOT 1
- Positive paper sampling HOT 2
- Matching articles from SPECTER's dataset with S2ORC IDs HOT 1
- Using trained model: which tokenizer?
- where can I get the dataset used in the paper? HOT 1
- ERROR: Allennlp params.py ValueError: Cannot convert variable to bool: all
- Does the model use Bert's vocabulary or scibert's vocabulary, or do they all use it? Where is it used HOT 1
- How to create the vocab files? (tokens.txt, non_padded_namespaces.txt, venue.txt) HOT 5
- RoBERTa instead of BERT: what changes in SPECTER scripts?
- concat_title_abstract must be True in scripts/pytorch_lightning_training_script/train.py#L73 HOT 1
- Format specified for json file containing title and abstract is wrong.
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from specter.