Token annotation error for XML output with non-standard rules

[marijnschraagen]

Token annotation error for XML output with non-standard rules

[proycon]

I can indeed replicate this. It seems related to LanguageMachines/ucto#72 .

[kosloot]

Well.... The problem is here that frog uses the 'language' nld-vnn which refers to the configuration in /usr/local/share/frog/nld-vnn/
Ucto is then initialized from /usr/local/share/frog/nld-vnn/frog.cfg using:

[[tokenizer]]
rulesFile=tokconfig-nld-historical

So for ucto the language is nld-historical

This is confusing for us as well the software....

When I run Frog like this:
frog -c /usr/local/share/frog/nld-vnn/frog.cfg -X uit.xml -t txt
all seem well.
So that might be a quick workaround.

As a matter of fact, I am inclined to think that this is an abuse of the --language parameter.
It is meant to give frog a hint about the languages to detect, and NOT to tell which configuration to use.

When using --languages, frog should ignore the rulesFile information from the frog config file.
This was so until @proycon "fixed" it in #80
That was putting the cart before the horse probably.

We need to rethink this.

[kosloot]

fixed according to #80