kanasimi/chinese_converter

繁簡轉換函式庫 追求正確率 先解析詞性再繁簡轉換 繁體中文↔簡體中文轉換 Chinese converter between Traditional Chinese and Simplified Chinese.

chinese_converter's Issues

無法安裝和執行 LTP server

系統： Windows 10, Python 3.12.3

已安裝以下套件:

fire                      0.6.0
pip                       24.0
torch                     2.3.0
tornado                   6.4

無法按照 README 提供的指令安裝 LTP；

D:\*****\test\python\ltp>pip install --force-reinstall "ltp==4.1.5.post2"
Collecting ltp==4.1.5.post2
  Using cached ltp-4.1.5.post2-py3-none-any.whl.metadata (4.8 kB)
Collecting torch>=1.2.0 (from ltp==4.1.5.post2)
  Using cached torch-2.3.0-cp312-cp312-win_amd64.whl.metadata (26 kB)
Collecting transformers<=4.7.0,>=4.0.0 (from ltp==4.1.5.post2)
  Using cached transformers-4.7.0-py3-none-any.whl.metadata (48 kB)
Collecting pygtrie<2.5,>=2.3.0 (from ltp==4.1.5.post2)
  Using cached pygtrie-2.4.2-py3-none-any.whl
Collecting packaging>=20.0 (from ltp==4.1.5.post2)
  Using cached packaging-24.0-py3-none-any.whl.metadata (3.2 kB)
Collecting filelock (from torch>=1.2.0->ltp==4.1.5.post2)
  Using cached filelock-3.14.0-py3-none-any.whl.metadata (2.8 kB)
Collecting typing-extensions>=4.8.0 (from torch>=1.2.0->ltp==4.1.5.post2)
  Using cached typing_extensions-4.12.0-py3-none-any.whl.metadata (3.0 kB)
Collecting sympy (from torch>=1.2.0->ltp==4.1.5.post2)
  Using cached sympy-1.12.1-py3-none-any.whl.metadata (12 kB)
Collecting networkx (from torch>=1.2.0->ltp==4.1.5.post2)
  Using cached networkx-3.3-py3-none-any.whl.metadata (5.1 kB)
Collecting jinja2 (from torch>=1.2.0->ltp==4.1.5.post2)
  Using cached jinja2-3.1.4-py3-none-any.whl.metadata (2.6 kB)
Collecting fsspec (from torch>=1.2.0->ltp==4.1.5.post2)
  Using cached fsspec-2024.5.0-py3-none-any.whl.metadata (11 kB)
Collecting mkl<=2021.4.0,>=2021.1.1 (from torch>=1.2.0->ltp==4.1.5.post2)
  Using cached mkl-2021.4.0-py2.py3-none-win_amd64.whl.metadata (1.4 kB)
Collecting huggingface-hub==0.0.8 (from transformers<=4.7.0,>=4.0.0->ltp==4.1.5.post2)
  Using cached huggingface_hub-0.0.8-py3-none-any.whl.metadata (8.7 kB)
Collecting numpy>=1.17 (from transformers<=4.7.0,>=4.0.0->ltp==4.1.5.post2)
  Using cached numpy-1.26.4-cp312-cp312-win_amd64.whl.metadata (61 kB)
Collecting pyyaml (from transformers<=4.7.0,>=4.0.0->ltp==4.1.5.post2)
  Using cached PyYAML-6.0.1-cp312-cp312-win_amd64.whl.metadata (2.1 kB)
Collecting regex!=2019.12.17 (from transformers<=4.7.0,>=4.0.0->ltp==4.1.5.post2)
  Using cached regex-2024.5.15-cp312-cp312-win_amd64.whl.metadata (41 kB)
Collecting requests (from transformers<=4.7.0,>=4.0.0->ltp==4.1.5.post2)
  Using cached requests-2.32.3-py3-none-any.whl.metadata (4.6 kB)
Collecting sacremoses (from transformers<=4.7.0,>=4.0.0->ltp==4.1.5.post2)
  Using cached sacremoses-0.1.1-py3-none-any.whl.metadata (8.3 kB)
Collecting tokenizers<0.11,>=0.10.1 (from transformers<=4.7.0,>=4.0.0->ltp==4.1.5.post2)
  Using cached tokenizers-0.10.3.tar.gz (212 kB)
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'done'
Collecting tqdm>=4.27 (from transformers<=4.7.0,>=4.0.0->ltp==4.1.5.post2)
  Using cached tqdm-4.66.4-py3-none-any.whl.metadata (57 kB)
Collecting intel-openmp==2021.* (from mkl<=2021.4.0,>=2021.1.1->torch>=1.2.0->ltp==4.1.5.post2)
  Using cached intel_openmp-2021.4.0-py2.py3-none-win_amd64.whl.metadata (1.2 kB)
Collecting tbb==2021.* (from mkl<=2021.4.0,>=2021.1.1->torch>=1.2.0->ltp==4.1.5.post2)
  Using cached tbb-2021.12.0-py3-none-win_amd64.whl.metadata (1.1 kB)
Collecting colorama (from tqdm>=4.27->transformers<=4.7.0,>=4.0.0->ltp==4.1.5.post2)
  Using cached colorama-0.4.6-py2.py3-none-any.whl.metadata (17 kB)
Collecting MarkupSafe>=2.0 (from jinja2->torch>=1.2.0->ltp==4.1.5.post2)
  Using cached MarkupSafe-2.1.5-cp312-cp312-win_amd64.whl.metadata (3.1 kB)
Collecting charset-normalizer<4,>=2 (from requests->transformers<=4.7.0,>=4.0.0->ltp==4.1.5.post2)
  Using cached charset_normalizer-3.3.2-cp312-cp312-win_amd64.whl.metadata (34 kB)
Collecting idna<4,>=2.5 (from requests->transformers<=4.7.0,>=4.0.0->ltp==4.1.5.post2)
  Using cached idna-3.7-py3-none-any.whl.metadata (9.9 kB)
Collecting urllib3<3,>=1.21.1 (from requests->transformers<=4.7.0,>=4.0.0->ltp==4.1.5.post2)
  Using cached urllib3-2.2.1-py3-none-any.whl.metadata (6.4 kB)
Collecting certifi>=2017.4.17 (from requests->transformers<=4.7.0,>=4.0.0->ltp==4.1.5.post2)
  Using cached certifi-2024.2.2-py3-none-any.whl.metadata (2.2 kB)
Collecting click (from sacremoses->transformers<=4.7.0,>=4.0.0->ltp==4.1.5.post2)
  Using cached click-8.1.7-py3-none-any.whl.metadata (3.0 kB)
Collecting joblib (from sacremoses->transformers<=4.7.0,>=4.0.0->ltp==4.1.5.post2)
  Using cached joblib-1.4.2-py3-none-any.whl.metadata (5.4 kB)
Collecting mpmath<1.4.0,>=1.1.0 (from sympy->torch>=1.2.0->ltp==4.1.5.post2)
  Using cached mpmath-1.3.0-py3-none-any.whl.metadata (8.6 kB)
Using cached ltp-4.1.5.post2-py3-none-any.whl (94 kB)
Using cached packaging-24.0-py3-none-any.whl (53 kB)
Using cached torch-2.3.0-cp312-cp312-win_amd64.whl (159.7 MB)
Using cached transformers-4.7.0-py3-none-any.whl (2.5 MB)
Using cached huggingface_hub-0.0.8-py3-none-any.whl (34 kB)
Using cached mkl-2021.4.0-py2.py3-none-win_amd64.whl (228.5 MB)
Using cached intel_openmp-2021.4.0-py2.py3-none-win_amd64.whl (3.5 MB)
Using cached tbb-2021.12.0-py3-none-win_amd64.whl (286 kB)
Using cached numpy-1.26.4-cp312-cp312-win_amd64.whl (15.5 MB)
Using cached regex-2024.5.15-cp312-cp312-win_amd64.whl (268 kB)
Using cached tqdm-4.66.4-py3-none-any.whl (78 kB)
Using cached typing_extensions-4.12.0-py3-none-any.whl (37 kB)
Using cached filelock-3.14.0-py3-none-any.whl (12 kB)
Using cached fsspec-2024.5.0-py3-none-any.whl (316 kB)
Using cached jinja2-3.1.4-py3-none-any.whl (133 kB)
Using cached networkx-3.3-py3-none-any.whl (1.7 MB)
Using cached PyYAML-6.0.1-cp312-cp312-win_amd64.whl (138 kB)
Using cached requests-2.32.3-py3-none-any.whl (64 kB)
Using cached sacremoses-0.1.1-py3-none-any.whl (897 kB)
Using cached sympy-1.12.1-py3-none-any.whl (5.7 MB)
Using cached certifi-2024.2.2-py3-none-any.whl (163 kB)
Using cached charset_normalizer-3.3.2-cp312-cp312-win_amd64.whl (100 kB)
Using cached idna-3.7-py3-none-any.whl (66 kB)
Using cached MarkupSafe-2.1.5-cp312-cp312-win_amd64.whl (17 kB)
Using cached mpmath-1.3.0-py3-none-any.whl (536 kB)
Using cached urllib3-2.2.1-py3-none-any.whl (121 kB)
Using cached click-8.1.7-py3-none-any.whl (97 kB)
Using cached colorama-0.4.6-py2.py3-none-any.whl (25 kB)
Using cached joblib-1.4.2-py3-none-any.whl (301 kB)
Building wheels for collected packages: tokenizers
  Building wheel for tokenizers (pyproject.toml): started
  Building wheel for tokenizers (pyproject.toml): finished with status 'error'
  error: subprocess-exited-with-error
  
  Building wheel for tokenizers (pyproject.toml) did not run successfully.
  exit code: 1
  
  [51 lines of output]
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build\lib.win-amd64-cpython-312
  creating build\lib.win-amd64-cpython-312\tokenizers
  copying py_src\tokenizers\__init__.py -> build\lib.win-amd64-cpython-312\tokenizers
  creating build\lib.win-amd64-cpython-312\tokenizers\models
  copying py_src\tokenizers\models\__init__.py -> build\lib.win-amd64-cpython-312\tokenizers\models
  creating build\lib.win-amd64-cpython-312\tokenizers\decoders
  copying py_src\tokenizers\decoders\__init__.py -> build\lib.win-amd64-cpython-312\tokenizers\decoders
  creating build\lib.win-amd64-cpython-312\tokenizers\normalizers
  copying py_src\tokenizers\normalizers\__init__.py -> build\lib.win-amd64-cpython-312\tokenizers\normalizers
  creating build\lib.win-amd64-cpython-312\tokenizers\pre_tokenizers
  copying py_src\tokenizers\pre_tokenizers\__init__.py -> build\lib.win-amd64-cpython-312\tokenizers\pre_tokenizers
  creating build\lib.win-amd64-cpython-312\tokenizers\processors
  copying py_src\tokenizers\processors\__init__.py -> build\lib.win-amd64-cpython-312\tokenizers\processors
  creating build\lib.win-amd64-cpython-312\tokenizers\trainers
  copying py_src\tokenizers\trainers\__init__.py -> build\lib.win-amd64-cpython-312\tokenizers\trainers
  creating build\lib.win-amd64-cpython-312\tokenizers\implementations
  copying py_src\tokenizers\implementations\base_tokenizer.py -> build\lib.win-amd64-cpython-312\tokenizers\implementations
  copying py_src\tokenizers\implementations\bert_wordpiece.py -> build\lib.win-amd64-cpython-312\tokenizers\implementations
  copying py_src\tokenizers\implementations\byte_level_bpe.py -> build\lib.win-amd64-cpython-312\tokenizers\implementations
  copying py_src\tokenizers\implementations\char_level_bpe.py -> build\lib.win-amd64-cpython-312\tokenizers\implementations
  copying py_src\tokenizers\implementations\sentencepiece_bpe.py -> build\lib.win-amd64-cpython-312\tokenizers\implementations
  copying py_src\tokenizers\implementations\sentencepiece_unigram.py -> build\lib.win-amd64-cpython-312\tokenizers\implementations
  copying py_src\tokenizers\implementations\__init__.py -> build\lib.win-amd64-cpython-312\tokenizers\implementations
  creating build\lib.win-amd64-cpython-312\tokenizers\tools
  copying py_src\tokenizers\tools\visualizer.py -> build\lib.win-amd64-cpython-312\tokenizers\tools
  copying py_src\tokenizers\tools\__init__.py -> build\lib.win-amd64-cpython-312\tokenizers\tools
  copying py_src\tokenizers\__init__.pyi -> build\lib.win-amd64-cpython-312\tokenizers
  copying py_src\tokenizers\models\__init__.pyi -> build\lib.win-amd64-cpython-312\tokenizers\models
  copying py_src\tokenizers\decoders\__init__.pyi -> build\lib.win-amd64-cpython-312\tokenizers\decoders
  copying py_src\tokenizers\normalizers\__init__.pyi -> build\lib.win-amd64-cpython-312\tokenizers\normalizers
  copying py_src\tokenizers\pre_tokenizers\__init__.pyi -> build\lib.win-amd64-cpython-312\tokenizers\pre_tokenizers
  copying py_src\tokenizers\processors\__init__.pyi -> build\lib.win-amd64-cpython-312\tokenizers\processors
  copying py_src\tokenizers\trainers\__init__.pyi -> build\lib.win-amd64-cpython-312\tokenizers\trainers
  copying py_src\tokenizers\tools\visualizer-styles.css -> build\lib.win-amd64-cpython-312\tokenizers\tools
  running build_ext
  running build_rust
  error: can't find Rust compiler
  
  If you are using an outdated pip version, it is possible a prebuilt wheel is available for this package but pip is not able to install from it. Installing from the wheel would avoid the need for a Rust compiler.
  
  To update pip, run:
  
      pip install --upgrade pip
  
  and then retry package installation.
  
  If you did intend to build this package from source, try installing a Rust compiler from your system package manager and ensure it is on the PATH during installation. Alternatively, rustup (available at https://rustup.rs) is the recommended way to download and update the Rust compiler toolchain.
  [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for tokenizers
Failed to build tokenizers
ERROR: Could not build wheels for tokenizers, which is required to install pyproject.toml-based projects

如果用其他方式安裝以下套件：

ltp                       4.2.13
ltp-core                  0.1.4
ltp-extension             0.1.13

執行 LTP server 時仍會出錯：

D:\*****\test\python\ltp>python server.base.py serve
C:\Users\***\AppData\Local\Programs\Python\Python312\Lib\site-packages\huggingface_hub\file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
  warnings.warn(
Traceback (most recent call last):
  File "D:\*****\test\python\ltp\server.base.py", line 154, in <module>
    Fire(Server)
  File "C:\Users\***\AppData\Local\Programs\Python\Python312\Lib\site-packages\fire\core.py", line 143, in Fire
    component_trace = _Fire(component, args, parsed_flag_args, context, name)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\***\AppData\Local\Programs\Python\Python312\Lib\site-packages\fire\core.py", line 477, in _Fire
    component, remaining_args = _CallAndUpdateTrace(
                                ^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\***\AppData\Local\Programs\Python\Python312\Lib\site-packages\fire\core.py", line 693, in _CallAndUpdateTrace
    component = fn(*varargs, **kwargs)
                ^^^^^^^^^^^^^^^^^^^^^^
  File "D:\*****\test\python\ltp\server.base.py", line 52, in __init__
    self.ltp = LTP(path=path, device=device)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\***\AppData\Local\Programs\Python\Python312\Lib\site-packages\ltp\interface.py", line 125, in LTP
    return LTP_neural._from_pretrained(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\***\AppData\Local\Programs\Python\Python312\Lib\site-packages\ltp\nerual.py", line 534, in _from_pretrained
    ltp = cls(**model_kwargs).to(map_location)
          ^^^^^^^^^^^^^^^^^^^
TypeError: LTP.__init__() got an unexpected keyword argument 'path'