Comments (7)
If you share the full stack trace and the config file you used, we might also be able to help.
from openkiwi.
Hello @BigBorg!
Can you provide a reproducible example using public data?
Ideally, a config and a small dataset that encountered this error would be amazing.
I would love to help, but it's hard to diagnose if I can't reproduce the issue :)
from openkiwi.
Sorry the dataset is private. I might try some public dataset to see if this happens again.
from openkiwi.
Sending codes out from the company i work for is restricted. Turning off sentence-ll solves the problem. Is it possible that the error is 0 then becomes inf after log? Besides, sentence scores produced by the model might be larger than 1, how do I interpret score?
from openkiwi.
Understandable. It could indeed be the case, but it seems weird that we never encountered this error while training with our own data or with publicly available datasets... If the reason becomes clearer, please let us know.
On the second question, sentence scores are an attempt to predict TER (Translation Error Rate), or the distance that separates the current translation from a "perfect" translation. With 1 meaning the whole sentence needs to be changed and 0 meaning the sentence is correct.
The model shouldn't produce scores above 1, what kind of scores are you seeing? Are you sure your training data contains all TER values in the range [0-1]?
from openkiwi.
Thanks for reminding me to inspect training data. It does contain hter larger than 1. I don't know why tercom is producing such result. I might try python package pyter to generate hter.
from openkiwi.
Tercom computes hter as
(Edit Distance mt - pe ) / (len(pe))
Thus if the MT is longer than the postedition, you can have an hter longer than 1 (this will typically be a case of MT repetitions / hallucinations). In the QE shared task, the scores are truncated to be at most 1.
The sentence-scores output by the model can be greater than 1 if you turn off sentence-ll.
As you can see in the code, the sentence score prediction module does not have a squashing function in the last layer.
If you enable sentence-ll
, the model outputs a gaussian distribution that is truncated over the interval [0, 1]. In that case, model scores are the mean of that distribution, which will always lie within the interval itself.
from openkiwi.
Related Issues (20)
- TypeError: cannot unpack non-iterable NoneType object HOT 1
- The prediction process is not complete by Predictor Estimator. HOT 5
- OpenKiwi always download the tokenizer files for XLMRoberta even if a local path is configured. HOT 2
- Do openKiwi have confident score? HOT 1
- Error Pre-Training Predictor: "model -> encoder -> encode_source extra fields not permitted (type=value_error.extra)" HOT 1
- some confusions
- pkgutil.iter_modules() error: 'PosixPath' object has no attribute 'startswith'
- Got exception when import kiwi
- Seems that maximum token support for a sentence is 512?
- PicklingError: Can't pickle <class 'kiwi.data.encoders.wmt_qe_data_encoder.InputFields[PositiveInt]'>: attribute lookup InputFields[PositiveInt] on kiwi.data.encoders.wmt_qe_data_encoder failed HOT 2
- Do you need to tokenize your data when using a BERT/ROBERTA model?
- Pretrain config file
- What are source_pos and target_pos in the train_config.yaml?
- Why does it need "--model" paramter when I give a specific config? HOT 2
- What languages do the OpenKiwi support?
- some problems about data without alignments HOT 11
- I suppose that the code comment should be remove. HOT 2
- Error at Predictor Training: "Predictor is not a subclass of QESystem" HOT 2
- OSError: Can't load weights for 'xlm-roberta-base'. HOT 16
- open cannot unpack non-iterable NoneType object HOT 16
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from openkiwi.