winstarwang/rasa_nlu_bert

rasa_nlu with bert, base on rasa_nlu 0.14.4

rasa_nlu_bert's Issues

hi

你好，冒昧请问一下，如果我要要修改BERT做slot filling 和 intent detection，但是稍微改了一下bert模型，也可以在你这个上面跑吗

The tokenizer breaks ner_CRF because it doesn't set the `pattern` attribute

['can', "'", 't', 'find', 'it'] # result of message.get("tokens")
Traceback (most recent call last):
  File "src/training/train.py", line 48, in <module>
    trainer.train()
  File "src/training/train.py", line 25, in train
    self.trainer.train(training_data)
  File "/home/shail/natural-language-processing/venv/lib/python3.5/site-packages/rasa_nlu/model.py", line 196, in train
    **context)
  File "/home/shail/natural-language-processing/venv/lib/python3.5/site-packages/rasa_nlu/extractors/crf_entity_extractor.py", line 140, in train
    dataset = self._create_dataset(filtered_entity_examples)
  File "/home/shail/natural-language-processing/venv/lib/python3.5/site-packages/rasa_nlu/extractors/crf_entity_extractor.py", line 149, in _create_dataset
    dataset.append(self._from_json_to_crf(example, entity_offsets))
  File "/home/shail/natural-language-processing/venv/lib/python3.5/site-packages/rasa_nlu/extractors/crf_entity_extractor.py", line 468, in _from_json_to_crf
    return self._from_text_to_crf(message, ents)
  File "/home/shail/natural-language-processing/venv/lib/python3.5/site-packages/rasa_nlu/extractors/crf_entity_extractor.py", line 529, in _from_text_to_crf
    pattern = self.__pattern_of_token(message, i)
  File "/home/shail/natural-language-processing/venv/lib/python3.5/site-packages/rasa_nlu/extractors/crf_entity_extractor.py", line 508, in __pattern_of_token
    return c[i].get("pattern", {})
AttributeError: 'str' object has no attribute 'get'