Comments (4)
However, I'll take this chance to clarify some details about model re-training. I believe our documentation is slightly misleading as it is.
There are two ways to continue to train a model: 1 - Resuming and 2 - Loading
Resuming
When you are resuming, you are retrieving the previous state of the trainer. This includes everything related to the training process including, for example, optimizer parameters. The use case for this is things like preemptible servers where your training might stop and you might need to resume from the latest checkpoint. As such, you have to increase your total number of epochs if you want to continue training.
When resuming, Kiwi will first look for the trainer in your output-dir
and then in runs/uuid
if you don't specify the output-dir
. In your case, since you state the output-dir
there's no need to state the run-uuid
.
Loading
On the other hand, if you just want to load model weights and continue training with new optimizer/trainer parameters you can just pass the load-model
flag pointing at the model you'd like to load.
from openkiwi.
Hello @damyana79
Thanks for reporting this issue! I'll look into this and get back to you soon.
from openkiwi.
Hey @damyana79
I haven't been able to reproduce your issue with resuming. It seems to be happening because Kiwi cannot load the trainer and model from your previous run.
This means that for some reason it is looking in the wrong place. Would you mind sharing your output.log? This should be stored in the same folder as your model
from openkiwi.
This behaviour as been clarified and improved with #36.
I will now close this. Feel free to re-open if your problem continues!
from openkiwi.
Related Issues (20)
- TypeError: cannot unpack non-iterable NoneType object HOT 1
- The prediction process is not complete by Predictor Estimator. HOT 5
- OpenKiwi always download the tokenizer files for XLMRoberta even if a local path is configured. HOT 2
- Do openKiwi have confident score? HOT 1
- Error Pre-Training Predictor: "model -> encoder -> encode_source extra fields not permitted (type=value_error.extra)" HOT 1
- some confusions
- pkgutil.iter_modules() error: 'PosixPath' object has no attribute 'startswith'
- Got exception when import kiwi
- Seems that maximum token support for a sentence is 512?
- PicklingError: Can't pickle <class 'kiwi.data.encoders.wmt_qe_data_encoder.InputFields[PositiveInt]'>: attribute lookup InputFields[PositiveInt] on kiwi.data.encoders.wmt_qe_data_encoder failed HOT 2
- Do you need to tokenize your data when using a BERT/ROBERTA model?
- Pretrain config file
- What are source_pos and target_pos in the train_config.yaml?
- Why does it need "--model" paramter when I give a specific config? HOT 2
- What languages do the OpenKiwi support?
- some problems about data without alignments HOT 11
- I suppose that the code comment should be remove. HOT 2
- Error at Predictor Training: "Predictor is not a subclass of QESystem" HOT 2
- OSError: Can't load weights for 'xlm-roberta-base'. HOT 16
- open cannot unpack non-iterable NoneType object HOT 16
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from openkiwi.