deeppavlov.core.common.errors.ConfigError: 'Given fasttext model does NOT match fasttext model used previously to train loaded model'

[deeppavlov]

deeppavlov.core.common.errors.ConfigError: 'Given fasttext model does NOT match fasttext model used previously to train loaded model'

[dilyararimovna]

Could you, please, provide more info:
do you try to run pre-trained model?
which embedding model do you use?

[dixiematt8]

I followed the instructions and ran the following command...

python deep.py interactbot skills/go_bot/config.json -t <TELEGRAM_TOKEN>

[dilyararimovna]

This error arises when one trying to download pre-trained model but provides with another embedding file (not with which it was trained).

[dixiematt8]

This is the full message...

(deeppavlov) mdixie@my-machine:/n/w1-mdixie/DeepPavlov/deeppavlov$ python deep.py interactbot skills/go_bot/config.json -t 445864402:AAF1f9IlVwSez7EByRA10e9gBHF-CfThkdk
/home/mdixie/.conda/envs/deeppavlov/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: compiletime version 3.5 of module 'tensorflow.python.framework.fast_tensor_util' does not match runtime version 3.6
return f(*args, **kwds)
/home/mdixie/.conda/envs/deeppavlov/lib/python3.6/site-packages/h5py/init.py:36: FutureWarning: Conversion of the second argument of issubdtype from float to np.floating is deprecated. In future, it will be treated as np.float64 == np.dtype(float).type.
from ._conv import register_converters as _register_converters
Using TensorFlow backend.
[loading vocabulary from ../download/vocabs/word.dict]
[loading vocabulary from ../download/vocabs/token.dict]
[loading vocabulary from ../download/vocabs/tag.dict]
[loading vocabulary from ../download/vocabs/char.dict]
[loading vocabulary from ../download/vocabs/classes.dict]

:: initializing GoalOrientedBotNetwork from saved

:: restoring checkpoint from ../download/go_bot_rnn/model

session restored
/home/mdixie/.conda/envs/deeppavlov/lib/python3.6/site-packages/tensorflow/python/ops/gradients_impl.py:96: UserWarning: Converting sparse IndexedSlices to a dense Tensor of unknown shape. This may consume a large amount of memory.
"Converting sparse IndexedSlices to a dense Tensor of unknown shape. "
[ loading slot values from ../download/slots/slot_vals.json ]

:: initializing NerNetwork from saved

:: restoring checkpoint from ../download/ner/dstc_ner_model

[loading embeddings from ../download/embeddings/dstc2_fasttext_model_100.bin]
/home/mdixie/.conda/envs/deeppavlov/lib/python3.6/site-packages/deeppavlov-0.0.1-py3.6.egg/deeppavlov/core/models/serializable.py:48: UserWarning: No load path is set for NLTKTokenizer!
warn("No load path is set for {}!".format(self.class.name))

:: initializing KerasIntentModel from saved

[ loading weights from intent_cnn.h5 ]
Exception in <class 'deeppavlov.models.classifiers.intents.intent_model.KerasIntentModel'>
Traceback (most recent call last):
File "deep.py", line 63, in
main()
File "deep.py", line 55, in main
interact_model_by_telegram(pipeline_config_path, token)
File "/home/mdixie/.conda/envs/deeppavlov/lib/python3.6/site-packages/deeppavlov-0.0.1-py3.6.egg/telegram_utils/telegram_ui.py", line 57, in interact_model_by_telegram
model = build_model_from_config(config)
File "/home/mdixie/.conda/envs/deeppavlov/lib/python3.6/site-packages/deeppavlov-0.0.1-py3.6.egg/deeppavlov/core/commands/infer.py", line 34, in build_model_from_config
model = from_params(REGISTRY[model_name], model_config, vocabs=vocabs, mode=mode)
File "/home/mdixie/.conda/envs/deeppavlov/lib/python3.6/site-packages/deeppavlov-0.0.1-py3.6.egg/deeppavlov/core/common/params.py", line 49, in from_params
mode=kwargs['mode'])
File "/home/mdixie/.conda/envs/deeppavlov/lib/python3.6/site-packages/deeppavlov-0.0.1-py3.6.egg/deeppavlov/core/common/params.py", line 52, in from_params
model = cls(**dict(config_params, **kwargs))
File "/home/mdixie/.conda/envs/deeppavlov/lib/python3.6/site-packages/deeppavlov-0.0.1-py3.6.egg/deeppavlov/core/models/tf_backend.py", line 47, in call
obj.init(*args, **kwargs)
File "/home/mdixie/.conda/envs/deeppavlov/lib/python3.6/site-packages/deeppavlov-0.0.1-py3.6.egg/deeppavlov/core/models/tf_backend.py", line 28, in _wrapped
return func(*args, **kwargs)
File "/home/mdixie/.conda/envs/deeppavlov/lib/python3.6/site-packages/deeppavlov-0.0.1-py3.6.egg/deeppavlov/models/classifiers/intents/intent_model.py", line 138, in init
"Given fasttext model does NOT match fasttext model used previously to train loaded model")
deeppavlov.core.common.errors.ConfigError: 'Given fasttext model does NOT match fasttext model used previously to train loaded model'

[dilyararimovna]

This error arises when one tries to run pre-trained model with another embedding file (not this one with which it was trained).
Running the following command:

python deep.py interactbot skills/go_bot/config.json -t <TELEGRAM_TOKEN>

you are trying to run pre-trained models that previously should be downloaded within the following command:

python download.py

Did you run this command to download necessary files?

[dixiematt8]

Yes, I did as you said. It reads embedding from ../download/embeddings/dstc2_fasttext_model_100.bin
Just as you showed in the video.

[dilyararimovna]

Unfortunately, I tried to reproduce this mistake: I initialized new environment, installed library using instruction from the main README, run go_bot interaction but I didn't faced the error above.
You have to make sure you are using the same embedding file (it should be loaded), for that you can try to calculate md5 hash sum of embedding file using md5_hashsum function from

deeppavlov/models/classifiers/intents/utils.py

and compare it within those one that is given in
download/intents/intent_cnn_opt.json

[dixiematt8]

They are not matching. I got this bdfb6e757c6f7897c1d10022ea316ce5 using md5_hashsum from deeppavlov/models/classifiers/intents/utils.py. It is not matching with 39d91fd19b0f0240b8aa1e84b4d5fba4 from download/intents/intent_cnn_opt.json What should I do?

[dilyararimovna]

Could you, please, re-download model and embeddings?
For that, you have to delete download folder, and then run again:

python download.py

Maybe there were some troubles during downloading files.

[dixiematt8]

it fixed. But it is not starting on Telegram.

[seliverstov]

Can you provide a little bit more details about "it is not starting on Telegram"?

[dixiematt8]

so I gave Telegram token, and I open my Telegram bot and typed /start, but it is not saying "Welcome to the UI of ..."

[seliverstov]

Have you send to bot any other messages besides of "/start"? What was the response?

[dixiematt8]

I get no response for any message.

[seliverstov]

Try to update to new version 0.0.2.
Pull new code from master and run python setup.py develop
Re-download pre-trained models (old pre-trained models will not work)
Run command with your own telegram token python deep.py interactbot configs/go_bot/gobot_dstc2.json -t 464136058:BFGIezin812YaXVYdWhpgguQI0y123Cd0Vw

[dixiematt8]

You can close this. It works.