synkied/hanzipy

Hanzipy is a Chinese character and NLP module for Chinese language processing for python. It is primarily written to help provide a framework for Chinese language learners to explore Chinese.

hanzipy's Issues

Use cc_cedict_hsk_enhanced to allow more filters

https://github.com/Lingyuai/cc_cedict_hsk_enhanced

turn off DEBUG logging

it seems like one of the dependencies enables DEBUG logging. is there a way to prevent this?

UnicodeDecodeError when initializing HanziDecomposer

When I try to initialize HanziDecomposer, I get a UnicodeDecodeError. The error seems to stem from the init_decomposition() method when it tries to load a JSON file. Here's the traceback:

Traceback (most recent call last):
File "C:\MyPC\MyProject\HanziPyTest.py.py", line 12, in
decomposer = HanziDecomposer()
File "C:\MyPC\MyProject\venv\lib\site-packages\hanzipy\decomposer.py", line 23, in init
self.init_decomposition()
File "C:\MyPC\MyProject\venv\lib\site-packages\hanzipy\decomposer.py", line 52, in init_decomposition
self.radicals = json.load(radicals_file)
File "C:\MyPC\AppData\Local\Programs\Python\Python39\lib\json_init_.py", line 293, in load
return loads(fp.read(),
File "C:\MyPC\AppData\Local\Programs\Python\Python39\lib\encodings\cp1252.py", line 23, in decode
return codecs.charmap_decode(input,self.errors,decoding_table)[0]
UnicodeDecodeError: 'charmap' codec can't decode byte 0x8d in position 728: character maps to

Environment:

• OS: Windows11
• Python version: Python 3.9
• hanzipy version: 1.0.3

Steps to Reproduce:
Here's a minimal code snippet that reproduces the error:

from hanzipy import HanziDecomposer
decomposer = HanziDecomposer()

Phonetic regularities for traditional characters

dictionary.determine_phonetic_regularity('歡') returns a NoneType object, while dictionary.determine_phonetic_regularity('欢') does not. Is this behavior intentional?