cltl/aproof-icf-classifier

Classifier that can read medical reports and assign a functional level classification following the WHO ICF classification scheme.

aproof-icf-classifier's Issues

Add empty comment handling

The current code gives a 'Shape mismatch for blis.gemm: (0, 0) (480, 288) error in ln 121 of tok2vec.py: tokvecs = self.model.predict(docs)

Please consider adding support or more informative error handling (e.g. assert len(input)>0 'Encountered empty comment, please remove these in preprocessing' )

Running with local models from the hugging face .cache

Running with local models does not work even though the models are in the hugging face .cache:

/Users/piek/.cache/huggingface/transformers

E.g. there is. json file with the following info:

{"url": "https://huggingface.co/CLTL/icf-domains/resolve/main/special_tokens_map.json", "etag": ""e97d1993365bb21c88f390e8703e4c1af564821f"”}%

docker run -v /Users/piek/Desktop/r-COVID-19-TM/CovidHerstelZorg-ZonMW/data -e TRANSFORMERS_OFFLINE=1 piekvossen/a-proof-icf-classifier --in_csv example/input.csv --text_col text

Error message is below:

Loading input csv file: example/input.csv
Input csv file (example/input.csv) is successfuly loaded!
Anonymizing the text in "text" column. This might take a while.
Splitting the text in "text" column to sentences. This might take a while.
Generating domains predictions. This might take a while.
CUDA device not available; running on a CPU!
Cannot find the requested files in the cached path and outgoing traffic has been disabled. To enable model look-ups and downloads online, set 'local_files_only' to False.
Downloading the model from https://huggingface.co/CLTL/icf-domains
Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/transformers/configuration_utils.py", line 546, in get_config_dict
resolved_config_file = cached_path(
File "/usr/local/lib/python3.8/site-packages/transformers/file_utils.py", line 1402, in cached_path
output_path = get_from_cache(
File "/usr/local/lib/python3.8/site-packages/transformers/file_utils.py", line 1620, in get_from_cache
raise FileNotFoundError(
FileNotFoundError: Cannot find the requested files in the cached path and outgoing traffic has been disabled. To enable model look-ups and downloads online, set 'local_files_only' to False.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "./main.py", line 154, in
main(
File "./main.py", line 121, in main
sents['predictions'] = predict_domains(
File "/icfc/src/icf_classifiers.py", line 49, in predict_domains
model = MultiLabelClassificationModel(
File "/usr/local/lib/python3.8/site-packages/simpletransformers/classification/multi_label_classification_model.py", line 194, in init
self.config = config_class.from_pretrained(model_name, **self.args.config)
File "/usr/local/lib/python3.8/site-packages/transformers/configuration_utils.py", line 489, in from_pretrained
config_dict, kwargs = cls.get_config_dict(pretrained_model_name_or_path, **kwargs)
File "/usr/local/lib/python3.8/site-packages/transformers/configuration_utils.py", line 570, in get_config_dict
raise EnvironmentError(msg)
OSError: Can't load config for 'CLTL/icf-domains'. Make sure that:

• 'CLTL/icf-domains' is a correct model identifier listed on 'https://huggingface.co/models'

• or 'CLTL/icf-domains' is the correct path to a directory containing a config.json file