ZeroDivisionError: division by zero

[githubharald]

ZeroDivisionError: division by zero

[githubharald]

Maybe there are too few entries.
Please upload your words.txt so that I can have a look.

[githubharald]

so there are only 8 entries (lines)?
You need much more. A batch contains 50 samples (image-text pairs). 95% of the batches are used for training, 5% for evaluation (can be changed here).
When using this 95% / 5% split, we need at least 20 batches of data, that is at least 20*50=1000 samples. You can change the split to e.g. 90% / 10%, then 500 samples are enough.

Regarding the 'X' entries: the IAM format contains information which we don't need for this text recognition system. Therefore I just put non-sense values ('X') in these columns.
Instead of converting the data to IAM format, you could of course also change the DataLoader.py module to directly read your data. It's up to you - simply choose the solution which is easier or which better fits your needs.

[githubharald]

seems like the dataset contains 0 entries? This way, the total number of characters of your ground truth text is 0 and when computing the character error rate a division by 0 takes place.
See the README how the output should look like if the dataset contains entries:

---

Epoch: 1
Train NN
Batch: 1 / 500 Loss: 130.354
Batch: 2 / 500 Loss: 66.6619
...

Validate NN
Batch: 1 / 115
Ground truth -> Recognized
[OK] "," -> ","
[ERR:1] "Di" -> "D"
...

Character error rate: 13.956289%. Word accuracy: 67.721739%.

---

See this article section 1.2 on how to create a compatible dataset.

[helium390]

I'm using my own dataset (Korean letters) generated by you. Where is my mistake, I couldn't find. Should I modify the code in:
\# write filename, dummy-values and text
line = 'words-words-%d' % ctr + ' X X X X X X X ' + sample[0] + '\n'
But my 'words' file structure is same as given from website.

[helium390]

Yes, few entries, like:
a01-000u-00-00 X X X X X X X 개발과
a01-000u-00-01 X X X X X X X 제3항의
a01-000u-00-02 X X X X X X X 100일을.
a01-000u-00-03 X X X X X X X 자유와
a01-000u-00-04 X X X X X X X 25일부터
a01-000u-00-05 X X X X X X X 정치는
a01-000u-00-06 X X X X X X X 타인을
a01-000u-01-00 X X X X X X X 정책을
...............
But, I wanted to be sure that is working or not. I have another file that consists big entries. The 'X X X X X X' I didn't understand, really.

[helium390]

Thank you very much @githubharald . I got it!