Comments (4)
Now the code uses kaldialign to calculate insertions, deletions and substitutions. Hence, the generated files have now this information:
.../meeteval/example_files$ python -m meeteval.wer orcwer -h 'hyp*.stm' -r 'ref*.stm'
Wrote: .../hyp_orcwer_per_reco.json
Wrote: .../hyp_orcwer.json
.../meeteval/example_files$ cat hyp_orcwer.json
{
"errors": 18,
"length": 184,
"insertions": 0,
"deletions": 14,
"substitutions": 4,
"error_rate": 0.09782608695652174
}
.../meeteval/example_files$ cat hyp_orcwer_per_reco.json
{
"recordingA": {
"errors": 4,
"length": 124,
"insertions": 0,
"deletions": 0,
"substitutions": 4,
"error_rate": 0.03225806451612903,
"assignment": [
"Alice",
"Alice",
"Bob",
"Bob",
"Alice",
"Alice",
"Alice",
"Alice"
]
},
"recordingB": {
"errors": 14,
"length": 60,
"insertions": 0,
"deletions": 14,
"substitutions": 0,
"error_rate": 0.23333333333333334,
"assignment": [
"Bob",
"Bob",
"Alice",
"Alice"
]
}
Let me know, when you need more features or if a modification could simplify an integration in another framework.
from meeteval.
At the moment, it is not implemented. But we planned to integrate it. Since you are interested, I will take care to implement it.
Do you have a suggestion, how it should be obtained? ins/del/sub aren't unique values.
Do people expect to get the same as Kaldi?
I thought about adding my Kaldi wrapper, so that the optimization is done in python/cython, but the final WER is obtained with Kaldi (Editdistance is the same, but Kaldi additionally reports the ins/del/sub).
from meeteval.
One way would be to get the reference assignment
and then use the kaldialign package to compute the ins/del/sub. This is the WER package used in icefall, for example, and is kaldi-compatible.
from meeteval.
Thanks for the pointer, I forgot that kaldialign exists. The code will be much cleaner with kaldialign, then a wrapper around kaldi could be, since adding a pip dependency is simple.
from meeteval.
Related Issues (8)
- ImportError: ...cy_levenshtein.cpython-38-x86_64-linux-gnu.so: undefined symbol: _ZSt28__throw_bad_array_new_lengthv HOT 1
- Computing time-constrained WER HOT 7
- Missing `LICENSE` file HOT 2
- circular import error HOT 3
- ToDo: fix mimo_matching_v4 for python 3.9 HOT 3
- CTM files with comments can't be parsed HOT 1
- Does mdeval computes DER taking overlapped speech into account? HOT 9
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from meeteval.