tyxsspa/anytext

Official implementation code of the paper <AnyText: Multilingual Visual Text Generation And Editing>

anytext's Issues

pip安装包的时候总提示哈希值不匹配报错退出

Pip subprocess error:
ERROR: THESE PACKAGES DO NOT MATCH THE HASHES FROM THE REQUIREMENTS FILE. If you have updated the package versions, please update the hashes. Otherwise, examine the package contents carefully; someone may have tampered with them.
scipy from https://files.pythonhosted.org/packages/3a/98/3041681b3a0fef0f59033861a05d846aa1d284b07b531ce35b1b97cc2e93/scipy-1.11.3-cp310-cp310-win_amd64.whl (from albumentations==0.4.3->-r D:\AnyText\condaenv.9c8my0sr.requirements.txt (line 3)):
Expected sha256 033c3fd95d55012dd1148b201b72ae854d5086d25e7c316ec9850de4fe776929
Got 46e5279e43cda031d5e5bee91e28a25f9454440c7049eb110e290f1bbdc32146

failed

CondaEnvException: Pip failed

中文编码错误

Loaded model config from [models_yaml/anytext_sd15.yaml]
Loaded state_dict from [C:\Users\Admin.cache\modelscope\hub\damo\cv_anytext_text_generation_editing\anytext_v1.1.ckpt]
2024-01-05 16:11:55,194 - modelscope - INFO - initiate model from C:\Users\Admin.cache\modelscope\hub\damo\cv_anytext_text_generation_editing\nlp_csanmt_translation_zh2en
2024-01-05 16:11:55,194 - modelscope - INFO - initiate model from location C:\Users\Admin.cache\modelscope\hub\damo\cv_anytext_text_generation_editing\nlp_csanmt_translation_zh2en.
2024-01-05 16:11:55,196 - modelscope - INFO - initialize model from C:\Users\Admin.cache\modelscope\hub\damo\cv_anytext_text_generation_editing\nlp_csanmt_translation_zh2en
{'hidden_size': 1024, 'filter_size': 4096, 'num_heads': 16, 'num_encoder_layers': 24, 'num_decoder_layers': 6, 'attention_dropout': 0.0, 'residual_dropout': 0.0, 'relu_dropout': 0.0, 'layer_preproc': 'layer_norm', 'layer_postpro
c': 'none', 'shared_embedding_and_softmax_weights': True, 'shared_source_target_embedding': True, 'initializer_scale': 0.1, 'position_info_type': 'absolute', 'max_relative_dis': 16, 'num_semantic_encoder_layers': 4, 'src_vocab_size': 50000, 'trg_vocab_size': 50000, 'seed': 1234, 'beam_size': 4, 'lp_rate': 0.6, 'max_decoded_trg_len': 100, 'device_map': None, 'device': 'cuda'}
2024-01-05 16:11:55,202 - modelscope - WARNING - No val key and type key found in preprocessor domain of configuration.json file.
2024-01-05 16:11:55,202 - modelscope - WARNING - Cannot find available config to build preprocessor at mode inference, current config: {'src_lang': 'zh', 'tgt_lang': 'en', 'src_bpe': {'file': 'bpe.zh'}, 'model_dir': 'C:\Users\Admin\.cache\modelscope\hub\damo\cv_anytext_text_generation_editing\nlp_csanmt_translation_zh2en'}. trying to build by task and model information.
2024-01-05 16:11:55,202 - modelscope - WARNING - No preprocessor key ('csanmt-translation', 'translation') found in PREPROCESSOR_MAP, skip building preprocessor.
Traceback (most recent call last):
File "D:\AnyText\venv\lib\site-packages\modelscope\utils\registry.py", line 212, in build_from_cfg
return obj_cls(**args)
File "D:\AnyText\venv\lib\site-packages\modelscope\pipelines\nlp\translation_pipeline.py", line 54, in init
self._src_vocab = dict([
File "D:\AnyText\venv\lib\site-packages\modelscope\pipelines\nlp\translation_pipeline.py", line 54, in
self._src_vocab = dict([
UnicodeDecodeError: 'gbk' codec can't decode byte 0x84 in position 7: illegal multibyte sequence

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "D:\AnyText\venv\lib\site-packages\modelscope\utils\registry.py", line 212, in build_from_cfg
return obj_cls(**args)
File "C:\Users\Admin.cache\modelscope\modelscope_modules\cv_anytext_text_generation_editing\ms_wrapper.py", line 336, in init
pipe_model = AnyTextModel(model_dir=model, **kwargs)
File "C:\Users\Admin.cache\modelscope\modelscope_modules\cv_anytext_text_generation_editing\ms_wrapper.py", line 46, in init
self.init_model(**kwargs)
File "C:\Users\Admin.cache\modelscope\modelscope_modules\cv_anytext_text_generation_editing\ms_wrapper.py", line 240, in init_model
self.trans_pipe = pipeline(task=Tasks.translation, model=os.path.join(self.model_dir, 'nlp_csanmt_translation_zh2en'))
File "D:\AnyText\venv\lib\site-packages\modelscope\pipelines\builder.py", line 170, in pipeline
return build_pipeline(cfg, task_name=task)
File "D:\AnyText\venv\lib\site-packages\modelscope\pipelines\builder.py", line 65, in build_pipeline
return build_from_cfg(
File "D:\AnyText\venv\lib\site-packages\modelscope\utils\registry.py", line 215, in build_from_cfg
raise type(e)(f'{obj_cls.name}: {e}')
TypeError: function takes exactly 5 arguments (1 given)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "D:\AnyText\inference.py", line 3, in
pipe = pipeline('my-anytext-task', model='damo/cv_anytext_text_generation_editing', model_revision='v1.1.1')
File "D:\AnyText\venv\lib\site-packages\modelscope\pipelines\builder.py", line 170, in pipeline
return build_pipeline(cfg, task_name=task)
File "D:\AnyText\venv\lib\site-packages\modelscope\pipelines\builder.py", line 65, in build_pipeline
return build_from_cfg(
File "D:\AnyText\venv\lib\site-packages\modelscope\utils\registry.py", line 215, in build_from_cfg
raise type(e)(f'{obj_cls.name}: {e}')
TypeError: AnyTextPipeline: function takes exactly 5 arguments (1 given)

中文提示词翻译模型使用CPU推理的方法

demo内置的翻译模块大约消耗4G显存，如果需要使用使用CPU推理，因官方尚未支持，需要手动修改modelscope中翻译模块的推理代码，通常位于如下类似位置(仅举例)：
/home/username/anaconda3/envs/anytext/lib/python3.10/site-packages/modelscope/pipelines/nlp/translation_pipeline.py
然后在TranslationPipeline的__init__()函数中添加如下代码：

...
tf_config.gpu_options.allow_growth = True
tf_config.device_count['GPU'] = 0  # <-- add here!
self._session = tf.Session(config=tf_config)
...

中文报错

Global seed set to 43304008
Traceback (most recent call last):
File "/root/anaconda3/envs/anytext/lib/python3.10/site-packages/gradio/queueing.py", line 407, in call_prediction
output = await route_utils.call_process_api(
File "/root/anaconda3/envs/anytext/lib/python3.10/site-packages/gradio/route_utils.py", line 226, in call_process_api
output = await app.get_blocks().process_api(
File "/root/anaconda3/envs/anytext/lib/python3.10/site-packages/gradio/blocks.py", line 1550, in process_api
result = await self.call_function(
File "/root/anaconda3/envs/anytext/lib/python3.10/site-packages/gradio/blocks.py", line 1185, in call_function
prediction = await anyio.to_thread.run_sync(
File "/root/anaconda3/envs/anytext/lib/python3.10/site-packages/anyio/to_thread.py", line 33, in run_sync
return await get_asynclib().run_sync_in_worker_thread(
File "/root/anaconda3/envs/anytext/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 877, in run_sync_in_worker_thread
return await future
File "/root/anaconda3/envs/anytext/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 807, in run
result = context.run(func, *args)
File "/root/anaconda3/envs/anytext/lib/python3.10/site-packages/gradio/utils.py", line 661, in wrapper
response = f(*args, **kwargs)
File "/root/AnyText/demo.py", line 205, in process
raise gr.Error(rtn_warning)
gradio.exceptions.Error: 'You have input Chinese prompt but the translator is not loaded!'

installed pytorch_lighting but ModuleNotFoundError: MyCustomPipeline: MyCustomModel: No module named 'pytorch_lightning.utilities.distributed'

ModuleNotFoundError: MyCustomPipeline: MyCustomModel: No module named 'pytorch_lightning.utilities.distributed'

I would say avoid using pytorch_lighting, this is the worst lib I have ever thing in python enviroment.

TypeError: AnyTextPipeline: function takes exactly 5 arguments (1 given)

本地操作系统：Windows11

我的CUDA版本：11.8

我做了什么：
1.启动Anaconda虚拟环境anytext
2.在项目文件夹目录下输入python inference.py

终端命令行日志：
(anytext) PS D:\GitHubProject\AnyText> python .\inference.py
2024-01-11 17:27:52,120 - modelscope - INFO - PyTorch version 2.0.1+cu118 Found.
2024-01-11 17:27:52,123 - modelscope - INFO - TensorFlow version 2.13.0 Found.
2024-01-11 17:27:52,124 - modelscope - INFO - Loading ast index from C:\Users\92923.cache\modelscope\ast_indexer
2024-01-11 17:27:52,753 - modelscope - INFO - Loading done! Current index file version is 1.10.0, with md5 dece09f2ebbe99e0a53c20441372b40d and a total number of 946 components indexed
2024-01-11 17:27:57,202 - modelscope - INFO - Use user-specified model revision: v1.1.1
2024-01-11 17:28:12,416 - modelscope - WARNING - ('PIPELINES', 'my-anytext-task', 'anytext-pipeline') not found in ast index file
A matching Triton is not available, some optimizations will not be enabled.
Error caught was: No module named 'triton'
ControlLDM: Running in eps-prediction mode
Setting up MemoryEfficientCrossAttention. Query dim is 320, context_dim is None and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 320, context_dim is 768 and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 320, context_dim is None and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 320, context_dim is 768 and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 640, context_dim is None and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 640, context_dim is 768 and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 640, context_dim is None and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 640, context_dim is 768 and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is None and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is 768 and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is None and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is 768 and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is None and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is 768 and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is None and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is 768 and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is None and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is 768 and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is None and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is 768 and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 640, context_dim is None and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 640, context_dim is 768 and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 640, context_dim is None and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 640, context_dim is 768 and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 640, context_dim is None and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 640, context_dim is 768 and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 320, context_dim is None and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 320, context_dim is 768 and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 320, context_dim is None and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 320, context_dim is 768 and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 320, context_dim is None and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 320, context_dim is 768 and using 8 heads.
DiffusionWrapper has 859.52 M params.
making attention of type 'vanilla-xformers' with 512 in_channels
building MemoryEfficientAttnBlock with 512 in_channels...
Working with z of shape (1, 4, 32, 32) = 4096 dimensions.
making attention of type 'vanilla-xformers' with 512 in_channels
building MemoryEfficientAttnBlock with 512 in_channels...
Setting up MemoryEfficientCrossAttention. Query dim is 320, context_dim is None and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 320, context_dim is 768 and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 320, context_dim is None and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 320, context_dim is 768 and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 640, context_dim is None and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 640, context_dim is 768 and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 640, context_dim is None and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 640, context_dim is 768 and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is None and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is 768 and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is None and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is 768 and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is None and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is 768 and using 8 heads.
Loaded model config from [models_yaml/anytext_sd15.yaml]
Loaded state_dict from [C:\Users\92923.cache\modelscope\hub\damo\cv_anytext_text_generation_editing\anytext_v1.1.ckpt]
2024-01-11 17:28:32,619 - modelscope - INFO - initiate model from C:\Users\92923.cache\modelscope\hub\damo\cv_anytext_text_generation_editing\nlp_csanmt_translation_zh2en
2024-01-11 17:28:32,619 - modelscope - INFO - initiate model from location C:\Users\92923.cache\modelscope\hub\damo\cv_anytext_text_generation_editing\nlp_csanmt_translation_zh2en.
2024-01-11 17:28:32,622 - modelscope - INFO - initialize model from C:\Users\92923.cache\modelscope\hub\damo\cv_anytext_text_generation_editing\nlp_csanmt_translation_zh2en
{'hidden_size': 1024, 'filter_size': 4096, 'num_heads': 16, 'num_encoder_layers': 24, 'num_decoder_layers': 6, 'attention_dropout': 0.0, 'residual_dropout': 0.0, 'relu_dropout': 0.0, 'layer_preproc': 'layer_norm', 'layer_postproc': 'none', 'shared_embedding_and_softmax_weights': True, 'shared_source_target_embedding': True, 'initializer_scale': 0.1, 'position_info_type': 'absolute', 'max_relative_dis': 16, 'num_semantic_encoder_layers': 4, 'src_vocab_size': 50000, 'trg_vocab_size': 50000, 'seed': 1234, 'beam_size': 4, 'lp_rate': 0.6, 'max_decoded_trg_len': 100, 'device_map': None, 'device': 'cuda'}
2024-01-11 17:28:32,656 - modelscope - WARNING - No val key and type key found in preprocessor domain of configuration.json file.
2024-01-11 17:28:32,657 - modelscope - WARNING - Cannot find available config to build preprocessor at mode inference, current config: {'src_lang': 'zh', 'tgt_lang': 'en', 'src_bpe': {'file': 'bpe.zh'}, 'model_dir': 'C:\Users\92923\.cache\modelscope\hub\damo\cv_anytext_text_generation_editing\nlp_csanmt_translation_zh2en'}. trying to build by task and model information.
2024-01-11 17:28:32,657 - modelscope - WARNING - No preprocessor key ('csanmt-translation', 'translation') found in PREPROCESSOR_MAP, skip building preprocessor.
Traceback (most recent call last):
File "D:\Anaconda3\envs\anytext\lib\site-packages\modelscope\utils\registry.py", line 212, in build_from_cfg
return obj_cls(**args)
File "D:\Anaconda3\envs\anytext\lib\site-packages\modelscope\pipelines\nlp\translation_pipeline.py", line 54, in init
self._src_vocab = dict([
File "D:\Anaconda3\envs\anytext\lib\site-packages\modelscope\pipelines\nlp\translation_pipeline.py", line 54, in
self._src_vocab = dict([
UnicodeDecodeError: 'gbk' codec can't decode byte 0x84 in position 7: illegal multibyte sequence

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "D:\Anaconda3\envs\anytext\lib\site-packages\modelscope\utils\registry.py", line 212, in build_from_cfg
return obj_cls(**args)
File "C:\Users\92923.cache\modelscope\modelscope_modules\cv_anytext_text_generation_editing\ms_wrapper.py", line 336, in init
pipe_model = AnyTextModel(model_dir=model, **kwargs)
File "C:\Users\92923.cache\modelscope\modelscope_modules\cv_anytext_text_generation_editing\ms_wrapper.py", line 46, in init
self.init_model(**kwargs)
File "C:\Users\92923.cache\modelscope\modelscope_modules\cv_anytext_text_generation_editing\ms_wrapper.py", line 240, in init_model
self.trans_pipe = pipeline(task=Tasks.translation, model=os.path.join(self.model_dir, 'nlp_csanmt_translation_zh2en'))
File "D:\Anaconda3\envs\anytext\lib\site-packages\modelscope\pipelines\builder.py", line 170, in pipeline
return build_pipeline(cfg, task_name=task)
File "D:\Anaconda3\envs\anytext\lib\site-packages\modelscope\pipelines\builder.py", line 65, in build_pipeline
return build_from_cfg(
File "D:\Anaconda3\envs\anytext\lib\site-packages\modelscope\utils\registry.py", line 215, in build_from_cfg
raise type(e)(f'{obj_cls.name}: {e}')
TypeError: function takes exactly 5 arguments (1 given)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "D:\GitHubProject\AnyText\inference.py", line 3, in
pipe = pipeline('my-anytext-task', model='damo/cv_anytext_text_generation_editing', model_revision='v1.1.1')
File "D:\Anaconda3\envs\anytext\lib\site-packages\modelscope\pipelines\builder.py", line 170, in pipeline
return build_pipeline(cfg, task_name=task)
File "D:\Anaconda3\envs\anytext\lib\site-packages\modelscope\pipelines\builder.py", line 65, in build_pipeline
return build_from_cfg(
File "D:\Anaconda3\envs\anytext\lib\site-packages\modelscope\utils\registry.py", line 215, in build_from_cfg
raise type(e)(f'{obj_cls.name}: {e}')
TypeError: AnyTextPipeline: function takes exactly 5 arguments (1 given)

按照官方流程安装，报错了

2023-12-29 15:19:56,915 - modelscope - WARNING - ('PIPELINES', 'my-anytext-task', 'my-custom-pipeline') not found in ast index file
2023-12-29 15:19:56,915 - modelscope - INFO - initiate model from C:\Users\sunny.cache\modelscope\hub\damo\cv_anytext_text_generation_editing
2023-12-29 15:19:56,915 - modelscope - INFO - initiate model from location C:\Users\sunny.cache\modelscope\hub\damo\cv_anytext_text_generation_editing.
2023-12-29 15:19:56,916 - modelscope - INFO - initialize model from C:\Users\sunny.cache\modelscope\hub\damo\cv_anytext_text_generation_editing
2023-12-29 15:19:56,919 - modelscope - WARNING - ('MODELS', 'my-anytext-task', 'my-custom-model') not found in ast index file
WARNING[XFORMERS]: xFormers can't load C++/CUDA extensions. xFormers was built for:
PyTorch 2.0.1+cu118 with CUDA 1108 (you have 2.0.1+cpu)
Python 3.10.11 (you have 3.10.6)
Please reinstall xformers (see https://github.com/facebookresearch/xformers#installing-xformers)
Memory-efficient attention, SwiGLU, sparse and more won't be available.
Set XFORMERS_MORE_DETAILS=1 for more details
OMP: Error #15: Initializing libiomp5md.dll, but found libiomp5md.dll already initialized.
OMP: Hint This means that multiple copies of the OpenMP runtime have been linked into the program. That is dangerous, since it can degrade performance or cause incorrect results. The best thing to do is to ensure that only a single OpenMP runtime is linked into the process, e.g. by avoiding static linking of the OpenMP runtime in any library. As an unsafe, unsupported, undocumented workaround you can set the environment variable KMP_DUPLICATE_LIB_OK=TRUE to allow the program to continue to execute, but that may cause crashes or silently produce incorrect results. For more information, please see http://www.intel.com/software/products/support/.

按照官方流程安装，报错了

When release pretrained controlnet ckpts which can be used in SD1.5

can you provide a demo for try?

AnyTextPipeline: ResourceExhaustedError.__init__() missing 2 required positional arguments: 'op' and 'message

2024-01-09 20:22:31,137 - modelscope - INFO - PyTorch version 2.0.1 Found.
2024-01-09 20:22:31,152 - modelscope - INFO - TensorFlow version 2.13.0 Found.
2024-01-09 20:22:31,152 - modelscope - INFO - Loading ast index from /home/flyang/.cache/modelscope/ast_indexer
2024-01-09 20:22:34,583 - modelscope - INFO - Loading done! Current index file version is 1.10.0, with md5 c3f4ad4f45046b1c409c0f21a503852b and a total number of 946 components indexed

raceback (most recent call last):
File "/mnt/p/home/flyang/AnyText/demo.py", line 50, in
inference = pipeline('my-anytext-task', model='damo/cv_anytext_text_generation_editing', model_revision='v1.1.1', use_fp16=not args.use_fp32, use_translator=not args.no_translator, font_path=args.font_path)
File "/home/flyang/.local/lib/python3.10/site-packages/modelscope/pipelines/builder.py", line 170, in pipeline
return build_pipeline(cfg, task_name=task)
File "/home/flyang/.local/lib/python3.10/site-packages/modelscope/pipelines/builder.py", line 65, in build_pipeline
return build_from_cfg(
File "/home/flyang/.local/lib/python3.10/site-packages/modelscope/utils/registry.py", line 215, in build_from_cfg
raise type(e)(f'{obj_cls.name}: {e}')
TypeError: AnyTextPipeline: ResourceExhaustedError.init() missing 2 required positional arguments: 'op' and 'message

RuntimeError: expected scalar type Half but found Float and unexpected keyword argument '_js

搞定了

TypeError: MyCustomPipeline: MyCustomModel: function takes exactly 5 arguments (1 given)

运行demo.py报错无法启动
os环境：win11
设备：RTX4090
驱动版本：536.40
使用anaconda按照readme步骤安装
使用ttf：c:\windows\fonts\黑体 常规

可能相关的log：

Loaded model config from [models_yaml/anytext_sd15.yaml]
Loaded state_dict from [C:\Users\Joowa_AI\.cache\modelscope\hub\damo\cv_anytext_text_generation_editing\anytext_v1.1.ckpt]
2024-01-03 10:38:42,503 - modelscope - INFO - initiate model from C:\Users\Joowa_AI\.cache\modelscope\hub\damo\cv_anytext_text_generation_editing\nlp_csanmt_translation_zh2en
2024-01-03 10:38:42,504 - modelscope - INFO - initiate model from location C:\Users\Joowa_AI\.cache\modelscope\hub\damo\cv_anytext_text_generation_editing\nlp_csanmt_translation_zh2en.
2024-01-03 10:38:42,506 - modelscope - INFO - initialize model from C:\Users\Joowa_AI\.cache\modelscope\hub\damo\cv_anytext_text_generation_editing\nlp_csanmt_translation_zh2en
{'hidden_size': 1024, 'filter_size': 4096, 'num_heads': 16, 'num_encoder_layers': 24, 'num_decoder_layers': 6, 'attention_dropout': 0.0, 'residual_dropout': 0.0, 'relu_dropout': 0.0, 'layer_preproc': 'layer_norm', 'layer_postproc': 'none', 'shared_embedding_and_softmax_weights': True, 'shared_source_target_embedding': True, 'initializer_scale': 0.1, 'position_info_type': 'absolute', 'max_relative_dis': 16, 'num_semantic_encoder_layers': 4, 'src_vocab_size': 50000, 'trg_vocab_size': 50000, 'seed': 1234, 'beam_size': 4, 'lp_rate': 0.6, 'max_decoded_trg_len': 100, 'device_map': None, 'device': 'cuda'}
2024-01-03 10:38:42,515 - modelscope - WARNING - No val key and type key found in preprocessor domain of configuration.json file.
2024-01-03 10:38:42,515 - modelscope - WARNING - Cannot find available config to build preprocessor at mode inference, current config: {'src_lang': 'zh', 'tgt_lang': 'en', 'src_bpe': {'file': 'bpe.zh'}, 'model_dir': 'C:\\Users\\Joowa_AI\\.cache\\modelscope\\hub\\damo\\cv_anytext_text_generation_editing\\nlp_csanmt_translation_zh2en'}. trying to build by task and model information.
2024-01-03 10:38:42,516 - modelscope - WARNING - No preprocessor key ('csanmt-translation', 'translation') found in PREPROCESSOR_MAP, skip building preprocessor.
Traceback (most recent call last):
  File "G:\anaconda3\envs\anytext\lib\site-packages\modelscope\utils\registry.py", line 212, in build_from_cfg
    return obj_cls(**args)
  File "G:\anaconda3\envs\anytext\lib\site-packages\modelscope\pipelines\nlp\translation_pipeline.py", line 54, in __init__
    self._src_vocab = dict([
  File "G:\anaconda3\envs\anytext\lib\site-packages\modelscope\pipelines\nlp\translation_pipeline.py", line 54, in <listcomp>
    self._src_vocab = dict([
UnicodeDecodeError: 'gbk' codec can't decode byte 0x84 in position 7: illegal multibyte sequence

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "G:\anaconda3\envs\anytext\lib\site-packages\modelscope\utils\registry.py", line 210, in build_from_cfg
    return obj_cls._instantiate(**args)
  File "G:\anaconda3\envs\anytext\lib\site-packages\modelscope\models\base\base_model.py", line 67, in _instantiate
    return cls(**kwargs)
  File "C:\Users\Joowa_AI\.cache\modelscope\modelscope_modules\cv_anytext_text_generation_editing\ms_wrapper.py", line 43, in __init__
    self.init_model(**kwargs)
  File "C:\Users\Joowa_AI\.cache\modelscope\modelscope_modules\cv_anytext_text_generation_editing\ms_wrapper.py", line 225, in init_model
    self.trans_pipe = pipeline(task=Tasks.translation, model=os.path.join(self.model_dir, 'nlp_csanmt_translation_zh2en'))
  File "G:\anaconda3\envs\anytext\lib\site-packages\modelscope\pipelines\builder.py", line 170, in pipeline
    return build_pipeline(cfg, task_name=task)
  File "G:\anaconda3\envs\anytext\lib\site-packages\modelscope\pipelines\builder.py", line 65, in build_pipeline
    return build_from_cfg(
  File "G:\anaconda3\envs\anytext\lib\site-packages\modelscope\utils\registry.py", line 215, in build_from_cfg
    raise type(e)(f'{obj_cls.__name__}: {e}')
TypeError: function takes exactly 5 arguments (1 given)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "G:\anaconda3\envs\anytext\lib\site-packages\modelscope\utils\registry.py", line 212, in build_from_cfg
    return obj_cls(**args)
  File "C:\Users\Joowa_AI\.cache\modelscope\modelscope_modules\cv_anytext_text_generation_editing\ms_wrapper.py", line 320, in __init__
    super().__init__(model=model, auto_collate=False)
  File "G:\anaconda3\envs\anytext\lib\site-packages\modelscope\pipelines\base.py", line 99, in __init__
    self.model = self.initiate_single_model(model)
  File "G:\anaconda3\envs\anytext\lib\site-packages\modelscope\pipelines\base.py", line 53, in initiate_single_model
    return Model.from_pretrained(
  File "G:\anaconda3\envs\anytext\lib\site-packages\modelscope\models\base\base_model.py", line 183, in from_pretrained
    model = build_model(model_cfg, task_name=task_name)
  File "G:\anaconda3\envs\anytext\lib\site-packages\modelscope\models\builder.py", line 35, in build_model
    model = build_from_cfg(
  File "G:\anaconda3\envs\anytext\lib\site-packages\modelscope\utils\registry.py", line 215, in build_from_cfg
    raise type(e)(f'{obj_cls.__name__}: {e}')
TypeError: MyCustomModel: function takes exactly 5 arguments (1 given)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "G:\AnyText\demo.py", line 22, in <module>
    inference = pipeline('my-anytext-task', model='damo/cv_anytext_text_generation_editing', model_revision='v1.1.0')
  File "G:\anaconda3\envs\anytext\lib\site-packages\modelscope\pipelines\builder.py", line 170, in pipeline
    return build_pipeline(cfg, task_name=task)
  File "G:\anaconda3\envs\anytext\lib\site-packages\modelscope\pipelines\builder.py", line 65, in build_pipeline
    return build_from_cfg(
  File "G:\anaconda3\envs\anytext\lib\site-packages\modelscope\utils\registry.py", line 215, in build_from_cfg
    raise type(e)(f'{obj_cls.__name__}: {e}')
TypeError: MyCustomPipeline: MyCustomModel: function takes exactly 5 arguments (1 given)

conda list：
absl-py 2.0.0 pypi_0 pypi
addict 2.4.0 pypi_0 pypi
aiofiles 23.2.1 pypi_0 pypi
aiohttp 3.9.1 pypi_0 pypi
aiosignal 1.3.1 pypi_0 pypi
albumentations 0.4.3 pypi_0 pypi
aliyun-python-sdk-core 2.14.0 pypi_0 pypi
aliyun-python-sdk-kms 2.16.2 pypi_0 pypi
altair 4.2.2 pypi_0 pypi
antlr4-python3-runtime 4.9.3 pypi_0 pypi
anyio 3.7.1 pypi_0 pypi
astunparse 1.6.3 pypi_0 pypi
async-timeout 4.0.3 pypi_0 pypi
attrs 23.2.0 pypi_0 pypi
basicsr 1.4.2 pypi_0 pypi
blas 1.0 mkl
blinker 1.7.0 pypi_0 pypi
braceexpand 0.1.7 pypi_0 pypi
brotli-python 1.0.9 py310hd77b12b_7
bzip2 1.0.8 he774522_0
ca-certificates 2023.12.12 haa95532_0
cachetools 5.3.2 pypi_0 pypi
certifi 2023.11.17 py310haa95532_0
cffi 1.16.0 py310h2bbff1b_0
charset-normalizer 2.0.4 pyhd3eb1b0_0
click 8.1.7 pypi_0 pypi
cmake 3.28.1 pypi_0 pypi
colorama 0.4.6 pypi_0 pypi
contourpy 1.2.0 pypi_0 pypi
crcmod 1.7 pypi_0 pypi
cryptography 41.0.3 py310h3438e0d_0
cuda-cccl 12.3.101 0 nvidia
cuda-cudart 11.8.89 0 nvidia
cuda-cudart-dev 11.8.89 0 nvidia
cuda-cupti 11.8.87 0 nvidia
cuda-libraries 11.8.0 0 nvidia
cuda-libraries-dev 11.8.0 0 nvidia
cuda-nvrtc 11.8.89 0 nvidia
cuda-nvrtc-dev 11.8.89 0 nvidia
cuda-nvtx 11.8.86 0 nvidia
cuda-profiler-api 12.3.101 0 nvidia
cuda-runtime 11.8.0 0 nvidia
cudatoolkit 11.8.0 hd77b12b_0
cycler 0.12.1 pypi_0 pypi
cython 0.29.33 py310hd77b12b_0
datasets 2.14.7 pypi_0 pypi
dill 0.3.7 pypi_0 pypi
easydict 1.10 pypi_0 pypi
einops 0.4.1 pypi_0 pypi
entrypoints 0.4 pypi_0 pypi
exceptiongroup 1.2.0 pypi_0 pypi
fastapi 0.103.2 pypi_0 pypi
ffmpy 0.3.1 pypi_0 pypi
filelock 3.13.1 py310haa95532_0
flatbuffers 23.5.26 pypi_0 pypi
fonttools 4.47.0 pypi_0 pypi
freetype 2.12.1 ha860e81_0
frozenlist 1.4.1 pypi_0 pypi
fsspec 2023.10.0 pypi_0 pypi
ftfy 6.1.3 pypi_0 pypi
future 0.18.3 pypi_0 pypi
gast 0.4.0 pypi_0 pypi
giflib 5.2.1 h8cc25b3_3
gitdb 4.0.11 pypi_0 pypi
gitpython 3.1.40 pypi_0 pypi
gmpy2 2.1.2 py310h7f96b67_0
google-auth 2.25.2 pypi_0 pypi
google-auth-oauthlib 1.0.0 pypi_0 pypi
google-pasta 0.2.0 pypi_0 pypi
gradio 3.50.0 pypi_0 pypi
gradio-client 0.6.1 pypi_0 pypi
grpcio 1.60.0 pypi_0 pypi
h11 0.14.0 pypi_0 pypi
h5py 3.10.0 pypi_0 pypi
httpcore 1.0.2 pypi_0 pypi
httpx 0.26.0 pypi_0 pypi
huggingface-hub 0.17.3 pypi_0 pypi
idna 3.4 py310haa95532_0
imageio 2.9.0 pypi_0 pypi
imageio-ffmpeg 0.4.2 pypi_0 pypi
imgaug 0.2.6 pypi_0 pypi
importlib-metadata 7.0.1 pypi_0 pypi
importlib-resources 6.1.1 pypi_0 pypi
intel-openmp 2021.4.0 haa95532_3556
jieba 0.42.1 pypi_0 pypi
jinja2 3.1.2 py310haa95532_0
jmespath 0.10.0 pypi_0 pypi
joblib 1.3.2 pypi_0 pypi
jpeg 9e h2bbff1b_1
jsonschema 4.20.0 pypi_0 pypi
jsonschema-specifications 2023.12.1 pypi_0 pypi
keras 2.13.1 pypi_0 pypi
kiwisolver 1.4.5 pypi_0 pypi
kornia 0.6.7 pypi_0 pypi
lazy-loader 0.3 pypi_0 pypi
lerc 3.0 hd77b12b_0
libclang 16.0.6 pypi_0 pypi
libcublas 11.11.3.6 0 nvidia
libcublas-dev 11.11.3.6 0 nvidia
libcufft 10.9.0.58 0 nvidia
libcufft-dev 10.9.0.58 0 nvidia
libcurand 10.3.4.107 0 nvidia
libcurand-dev 10.3.4.107 0 nvidia
libcusolver 11.4.1.48 0 nvidia
libcusolver-dev 11.4.1.48 0 nvidia
libcusparse 11.7.5.86 0 nvidia
libcusparse-dev 11.7.5.86 0 nvidia
libdeflate 1.17 h2bbff1b_1
libffi 3.4.4 hd77b12b_0
libnpp 11.8.0.86 0 nvidia
libnpp-dev 11.8.0.86 0 nvidia
libnvjpeg 11.9.0.86 0 nvidia
libnvjpeg-dev 11.9.0.86 0 nvidia
libpng 1.6.39 h8cc25b3_0
libtiff 4.5.1 hd77b12b_0
libuv 1.44.2 h2bbff1b_0
libwebp 1.3.2 hbc33d0d_0
libwebp-base 1.3.2 h2bbff1b_0
lmdb 1.4.1 pypi_0 pypi
lz4-c 1.9.4 h2bbff1b_0
markdown 3.5.1 pypi_0 pypi
markdown-it-py 3.0.0 pypi_0 pypi
markupsafe 2.1.3 py310h2bbff1b_0
matplotlib 3.8.2 pypi_0 pypi
mdurl 0.1.2 pypi_0 pypi
mkl 2021.4.0 haa95532_640
mkl-service 2.4.0 py310h2bbff1b_0
mkl_fft 1.3.1 py310ha0764ea_0
mkl_random 1.2.2 py310h4ed8f06_0
mock 5.1.0 pypi_0 pypi
modelscope 1.10.0 pypi_0 pypi
mpc 1.1.0 h7edee0f_1
mpfr 4.0.2 h62dcd97_1
mpir 3.0.0 hec2e145_1
mpmath 1.3.0 py310haa95532_0
multidict 6.0.4 pypi_0 pypi
multiprocess 0.70.15 pypi_0 pypi
mypy-extensions 1.0.0 pypi_0 pypi
networkx 3.1 py310haa95532_0
numpy 1.23.3 py310h60c9a35_0
numpy-base 1.23.3 py310h04254f7_0
oauthlib 3.2.2 pypi_0 pypi
omegaconf 2.2.3 pypi_0 pypi
open-clip-torch 2.7.0 pypi_0 pypi
opencv-python 4.7.0.72 pypi_0 pypi
opencv-python-headless 4.9.0.80 pypi_0 pypi
openjpeg 2.4.0 h4fc8c34_0
openssl 1.1.1w h2bbff1b_0
opt-einsum 3.3.0 pypi_0 pypi
orjson 3.9.10 pypi_0 pypi
oss2 2.18.4 pypi_0 pypi
packaging 23.2 pypi_0 pypi
pandas 1.5.3 pypi_0 pypi
pillow 9.5.0 pypi_0 pypi
pip 23.3.2 pypi_0 pypi
protobuf 3.20.3 pypi_0 pypi
pyarrow 14.0.2 pypi_0 pypi
pyarrow-hotfix 0.6 pypi_0 pypi
pyasn1 0.5.1 pypi_0 pypi
pyasn1-modules 0.3.0 pypi_0 pypi
pycparser 2.21 pyhd3eb1b0_0
pycryptodome 3.19.1 pypi_0 pypi
pydantic 1.10.13 pypi_0 pypi
pydeck 0.8.1b0 pypi_0 pypi
pydeprecate 0.3.1 pypi_0 pypi
pydub 0.25.1 pypi_0 pypi
pygments 2.17.2 pypi_0 pypi
pympler 1.0.1 pypi_0 pypi
pyopenssl 23.2.0 py310haa95532_0
pyparsing 3.1.1 pypi_0 pypi
pyre-extensions 0.0.29 pypi_0 pypi
pysocks 1.7.1 py310haa95532_0
python 3.10.6 hbb2ffb3_1
python-dateutil 2.8.2 pypi_0 pypi
python-multipart 0.0.6 pypi_0 pypi
pytorch 2.0.1 py3.10_cuda11.8_cudnn8_0 pytorch
pytorch-cuda 11.8 h24eeafa_5 pytorch
pytorch-lightning 1.5.0 pypi_0 pypi
pytorch-mutex 1.0 cuda pytorch
pytz 2023.3.post1 pypi_0 pypi
pywavelets 1.5.0 pypi_0 pypi
pyyaml 6.0.1 pypi_0 pypi
referencing 0.32.0 pypi_0 pypi
regex 2023.12.25 pypi_0 pypi
requests 2.31.0 py310haa95532_0
requests-oauthlib 1.3.1 pypi_0 pypi
rich 13.7.0 pypi_0 pypi
rpds-py 0.16.2 pypi_0 pypi
rsa 4.9 pypi_0 pypi
sacremoses 0.0.53 pypi_0 pypi
safetensors 0.4.0 pypi_0 pypi
scikit-image 0.20.0 pypi_0 pypi
scipy 1.11.4 pypi_0 pypi
semantic-version 2.10.0 pypi_0 pypi
semver 3.0.2 pypi_0 pypi
sentencepiece 0.1.99 pypi_0 pypi
setuptools 68.2.2 py310haa95532_0
simplejson 3.19.2 pypi_0 pypi
six 1.16.0 pyhd3eb1b0_1
smmap 5.0.1 pypi_0 pypi
sniffio 1.3.0 pypi_0 pypi
sortedcontainers 2.4.0 pypi_0 pypi
sqlite 3.41.2 h2bbff1b_0
starlette 0.27.0 pypi_0 pypi
streamlit 1.20.0 pypi_0 pypi
subword-nmt 0.3.8 pypi_0 pypi
sympy 1.12 py310haa95532_0
tb-nightly 2.16.0a20240102 pypi_0 pypi
tensorboard 2.13.0 pypi_0 pypi
tensorboard-data-server 0.7.2 pypi_0 pypi
tensorflow 2.13.0 pypi_0 pypi
tensorflow-estimator 2.13.0 pypi_0 pypi
tensorflow-intel 2.13.0 pypi_0 pypi
tensorflow-io-gcs-filesystem 0.31.0 pypi_0 pypi
termcolor 2.4.0 pypi_0 pypi
test-tube 0.7.5 pypi_0 pypi
tf-keras-nightly 2.16.0.dev2023123010 pypi_0 pypi
tifffile 2023.12.9 pypi_0 pypi
timm 0.6.7 pypi_0 pypi
tk 8.6.12 h2bbff1b_0
tokenizers 0.14.1 pypi_0 pypi
toml 0.10.2 pypi_0 pypi
toolz 0.12.0 pypi_0 pypi
torch 2.0.1 pypi_0 pypi
torchaudio 2.0.2 pypi_0 pypi
torchmetrics 0.11.4 pypi_0 pypi
torchvision 0.15.2 pypi_0 pypi
tqdm 4.66.1 pypi_0 pypi
transformers 4.34.1 pypi_0 pypi
triton 2.0.0 pypi_0 pypi
typing-extensions 4.5.0 pypi_0 pypi
typing-inspect 0.9.0 pypi_0 pypi
typing_extensions 4.7.1 py310haa95532_0
tzdata 2023.4 pypi_0 pypi
tzlocal 5.2 pypi_0 pypi
urllib3 1.26.18 py310haa95532_0
uvicorn 0.25.0 pypi_0 pypi
validators 0.22.0 pypi_0 pypi
vc 14.2 h21ff451_1
vs2015_runtime 14.27.29016 h5e58377_2
watchdog 3.0.0 pypi_0 pypi
webdataset 0.2.5 pypi_0 pypi
websockets 11.0.3 pypi_0 pypi
werkzeug 3.0.1 pypi_0 pypi
wheel 0.41.2 py310haa95532_0
win_inet_pton 1.1.0 py310haa95532_0
wrapt 1.16.0 pypi_0 pypi
xformers 0.0.20 pypi_0 pypi
xxhash 3.4.1 pypi_0 pypi
xz 5.4.5 h8cc25b3_0
yapf 0.32.0 pypi_0 pypi
yarl 1.9.4 pypi_0 pypi
zipp 3.17.0 pypi_0 pypi
zlib 1.2.13 h8cc25b3_0
zstd 1.5.5 hd43e919_0

有大佬帮忙看看怎么回事嘛，十分感谢

运行成功的朋友，你们用的cuda、cudnn、tensorflow的版本是多少

按照官方的方案已经安装没有报错，可以运行demo的页面，但点击推理时会报以下错误：
Global seed set to 81808278
Building prefix dict from the default dictionary ...
Loading model from cache /tmp/jieba.cache
Loading model cost 0.808 seconds.
Prefix dict has been built successfully.
WARNING:tensorflow:From /root/anaconda3/envs/anytext/lib/python3.10/site-packages/modelscope/utils/device.py:60: is_gpu_available (from tensorflow.python.framework.test_util) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.config.list_physical_devices('GPU') instead.
2024-01-06 13:43:02.314823: F ./tensorflow/core/util/gpu_launch_config.h:169] Check failed: err == cudaSuccess (302 vs. 0)
Aborted
怀疑是tf和cuda的版本不对。

我的版本如下：
3080ti
cuda 11.8
cudnn 8.5
tensorflow 2.13

ComfyUI

It would be great to see AnyText on ComfyUI by @comfyanonymous to unlock text generation and text editing workflows. AnyText can also be chained with ControlNet for finer control of image generation.

Respond with 👍 for a community vote.

excellent work！ how to train

Installation step by step

I'm no coder, so copying the first line into CMD didn't work. How can I install this step by step. A guide for a beginner with 0 knowledge in coding. Thank you

(I checked the installation process and didn't understand how to use it)

有做成SdWebUI插件的计划吗

蹲一个

when can we expect the open-sourcing of the AnyText code?

Traceback (most recent call last): File "/ofs/test/AnyText/demo.py", line 241, in <module> block.load(fn=None, TypeError: EventListener._setup.<locals>.event_trigger() got an unexpected keyword argument '_js'

Traceback (most recent call last):
File "/ofs/test/AnyText/demo.py", line 241, in
block.load(fn=None,
TypeError: EventListener._setup..event_trigger() got an unexpected keyword argument '_js'
有人遇到这个问题吗？

TypeError: MyCustomPipeline: MyCustomModel: DataLossError.__init__() missing 2 required positional arguments: 'op' and 'message'

Traceback (most recent call last):
File "/data/AnyText/inference.py", line 3, in
pipe = pipeline('my-anytext-task', model='damo/cv_anytext_text_generation_editing', model_revision='v1.1.0')
File "/data/AnyText/python/lib/python3.10/site-packages/modelscope/pipelines/builder.py", line 170, in pipeline
return build_pipeline(cfg, task_name=task)
File "/data/AnyText/python/lib/python3.10/site-packages/modelscope/pipelines/builder.py", line 65, in build_pipeline
return build_from_cfg(
File "/data/AnyText/python/lib/python3.10/site-packages/modelscope/utils/registry.py", line 215, in build_from_cfg
raise type(e)(f'{obj_cls.name}: {e}')
TypeError: MyCustomPipeline: MyCustomModel: DataLossError.init() missing 2 required positional arguments: 'op' and 'message'

Possibility of putting avaliable on hugginfaces

I would like to ask if possible that we have a avaliable model and demo space on HugginFaces.
But congrats, very interesting work!!

Got error with yaml, I need some help. Please help,thank you

(anytext) PS E:\AI_Tools\anytext> python inference.py
Traceback (most recent call last):
File "E:\AI_Tools\anytext\inference.py", line 1, in
from modelscope.pipelines import pipeline
File "C:\Users\hekan\anaconda3\envs\anytext\lib\site-packages\modelscope_init_.py", line 4, in
from modelscope.utils.import_utils import LazyImportModule
File "C:\Users\hekan\anaconda3\envs\anytext\lib\site-packages\modelscope\utils_init_.py", line 1, in
from .hub import create_model_if_not_exist, read_config
File "C:\Users\hekan\anaconda3\envs\anytext\lib\site-packages\modelscope\utils\hub.py", line 12, in
from modelscope.utils.config import Config
File "C:\Users\hekan\anaconda3\envs\anytext\lib\site-packages\modelscope\utils\config.py", line 22, in
from modelscope.utils.import_utils import import_modules_from_file
File "C:\Users\hekan\anaconda3\envs\anytext\lib\site-packages\modelscope\utils\import_utils.py", line 18, in
from modelscope.utils.ast_utils import (INDEX_KEY, MODULE_KEY, REQUIREMENT_KEY,
File "C:\Users\hekan\anaconda3\envs\anytext\lib\site-packages\modelscope\utils\ast_utils.py", line 16, in
from modelscope.fileio.file import LocalStorage
File "C:\Users\hekan\anaconda3\envs\anytext\lib\site-packages\modelscope\fileio_init_.py", line 4, in
from .io import dump, dumps, load
File "C:\Users\hekan\anaconda3\envs\anytext\lib\site-packages\modelscope\fileio\io.py", line 8, in
from .format import JsonHandler, YamlHandler
File "C:\Users\hekan\anaconda3\envs\anytext\lib\site-packages\modelscope\fileio\format_init_.py", line 5, in
from .yaml import YamlHandler
File "C:\Users\hekan\anaconda3\envs\anytext\lib\site-packages\modelscope\fileio\format\yaml.py", line 2, in
import yaml
ModuleNotFoundError: No module named 'yaml'

Only CPU How to run?

Having issue with "my-custom-pipeline"&"my-anytext-task", need some help,pls!!

(anytext) C:\Users\Administrator\AnyText>python inference.py
2024-01-05 20:24:42,259 - modelscope - INFO - PyTorch version 2.0.1 Found.
2024-01-05 20:24:42,261 - modelscope - INFO - TensorFlow version 2.13.0 Found.
2024-01-05 20:24:42,261 - modelscope - INFO - Loading ast index from C:\Users\Administrator.cache\modelscope\ast_indexer
2024-01-05 20:24:42,314 - modelscope - INFO - Loading done! Current index file version is 1.10.0, with md5 74ad7dd6e2e0ce75b5aa96a1876541a5 and a total number of 946 components indexed
2024-01-05 20:24:43,458 - modelscope - INFO - Use user-specified model revision: v1.1.0
2024-01-05 20:24:45,919 - modelscope - WARNING - ('PIPELINES', 'my-anytext-task', 'my-custom-pipeline') not found in ast index file
2024-01-05 20:24:45,919 - modelscope - INFO - initiate model from C:\Users\Administrator.cache\modelscope\hub\damo\cv_anytext_text_generation_editing
2024-01-05 20:24:45,919 - modelscope - INFO - initiate model from location C:\Users\Administrator.cache\modelscope\hub\damo\cv_anytext_text_generation_editing.
2024-01-05 20:24:45,921 - modelscope - INFO - initialize model from C:\Users\Administrator.cache\modelscope\hub\damo\cv_anytext_text_generation_editing
2024-01-05 20:24:45,923 - modelscope - WARNING - ('MODELS', 'my-anytext-task', 'my-custom-model') not found in ast index file
Traceback (most recent call last):
File "C:\ProgramData\miniconda3\envs\anytext\lib\site-packages\modelscope\utils\registry.py", line 210, in build_from_cfg
return obj_cls._instantiate(**args)
File "C:\ProgramData\miniconda3\envs\anytext\lib\site-packages\modelscope\models\base\base_model.py", line 67, in _instantiate
return cls(**kwargs)
File "C:\Users\Administrator.cache\modelscope\modelscope_modules\cv_anytext_text_generation_editing\ms_wrapper.py", line 43, in init
self.init_model(**kwargs)
File "C:\Users\Administrator.cache\modelscope\modelscope_modules\cv_anytext_text_generation_editing\ms_wrapper.py", line 218, in init_model
self.font = ImageFont.truetype(font_path, size=60)
File "C:\ProgramData\miniconda3\envs\anytext\lib\site-packages\PIL\ImageFont.py", line 996, in truetype
return freetype(font)
File "C:\ProgramData\miniconda3\envs\anytext\lib\site-packages\PIL\ImageFont.py", line 993, in freetype
return FreeTypeFont(font, size, index, encoding, layout_engine)
File "C:\ProgramData\miniconda3\envs\anytext\lib\site-packages\PIL\ImageFont.py", line 248, in init
self.font = core.getfont(
OSError: cannot open resource

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "C:\ProgramData\miniconda3\envs\anytext\lib\site-packages\modelscope\utils\registry.py", line 212, in build_from_cfg
return obj_cls(**args)
File "C:\Users\Administrator.cache\modelscope\modelscope_modules\cv_anytext_text_generation_editing\ms_wrapper.py", line 320, in init
super().init(model=model, auto_collate=False)
File "C:\ProgramData\miniconda3\envs\anytext\lib\site-packages\modelscope\pipelines\base.py", line 99, in init
self.model = self.initiate_single_model(model)
File "C:\ProgramData\miniconda3\envs\anytext\lib\site-packages\modelscope\pipelines\base.py", line 53, in initiate_single_model
return Model.from_pretrained(
File "C:\ProgramData\miniconda3\envs\anytext\lib\site-packages\modelscope\models\base\base_model.py", line 183, in from_pretrained
model = build_model(model_cfg, task_name=task_name)
File "C:\ProgramData\miniconda3\envs\anytext\lib\site-packages\modelscope\models\builder.py", line 35, in build_model
model = build_from_cfg(
File "C:\ProgramData\miniconda3\envs\anytext\lib\site-packages\modelscope\utils\registry.py", line 215, in build_from_cfg
raise type(e)(f'{obj_cls.name}: {e}')
OSError: MyCustomModel: cannot open resource

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "C:\Users\Administrator\AnyText\inference.py", line 3, in
pipe = pipeline('my-anytext-task', model='damo/cv_anytext_text_generation_editing', model_revision='v1.1.0')
File "C:\ProgramData\miniconda3\envs\anytext\lib\site-packages\modelscope\pipelines\builder.py", line 170, in pipeline
return build_pipeline(cfg, task_name=task)
File "C:\ProgramData\miniconda3\envs\anytext\lib\site-packages\modelscope\pipelines\builder.py", line 65, in build_pipeline
return build_from_cfg(
File "C:\ProgramData\miniconda3\envs\anytext\lib\site-packages\modelscope\utils\registry.py", line 215, in build_from_cfg
raise type(e)(f'{obj_cls.name}: {e}')
OSError: MyCustomPipeline: MyCustomModel: cannot open resource

Sincerely suggest: Do not force users on a certain old lib version

I kindly suggest upgrade your code to pytorch_lighthing 2.5 if you consist using that lib.

Old lib for Old version fixed it not a good, it would be better make code tracing latest of lib rather than force users change their enviroment.

本地部署后运行出错 OSError: AnyTextPipeline: cannot open resource

我的操作系统：Windows11

我的CUDA版本：11.1

我做了什么：
1.启动Anaconda虚拟环境anytext
2.在项目文件夹目录下输入python inference.py

终端命令行日志：
(anytext) D:\GitHubProject\AnyText>python inference.py
2024-01-11 13:38:36,229 - modelscope - INFO - PyTorch version 2.0.1 Found.
2024-01-11 13:38:36,287 - modelscope - INFO - TensorFlow version 2.13.0 Found.
2024-01-11 13:38:36,287 - modelscope - INFO - Loading ast index from C:\Users\92923.cache\modelscope\ast_indexer
2024-01-11 13:38:36,965 - modelscope - INFO - Loading done! Current index file version is 1.10.0, with md5 dece09f2ebbe99e0a53c20441372b40d and a total number of 946 components indexed
2024-01-11 13:38:45,385 - modelscope - INFO - Use user-specified model revision: v1.1.1
2024-01-11 13:39:03,504 - modelscope - WARNING - ('PIPELINES', 'my-anytext-task', 'anytext-pipeline') not found in ast index file
Traceback (most recent call last):
File "D:\Anaconda3\envs\anytext\lib\site-packages\modelscope\utils\registry.py", line 212, in build_from_cfg
return obj_cls(**args)
File "C:\Users\92923.cache\modelscope\modelscope_modules\cv_anytext_text_generation_editing\ms_wrapper.py", line 336, in init
pipe_model = AnyTextModel(model_dir=model, **kwargs)
File "C:\Users\92923.cache\modelscope\modelscope_modules\cv_anytext_text_generation_editing\ms_wrapper.py", line 46, in init
self.init_model(**kwargs)
File "C:\Users\92923.cache\modelscope\modelscope_modules\cv_anytext_text_generation_editing\ms_wrapper.py", line 229, in init_model
self.font = ImageFont.truetype(font_path, size=60)
File "D:\Anaconda3\envs\anytext\lib\site-packages\PIL\ImageFont.py", line 996, in truetype
return freetype(font)
File "D:\Anaconda3\envs\anytext\lib\site-packages\PIL\ImageFont.py", line 993, in freetype
return FreeTypeFont(font, size, index, encoding, layout_engine)
File "D:\Anaconda3\envs\anytext\lib\site-packages\PIL\ImageFont.py", line 248, in init
self.font = core.getfont(
OSError: cannot open resource

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "D:\GitHubProject\AnyText\inference.py", line 3, in
pipe = pipeline('my-anytext-task', model='damo/cv_anytext_text_generation_editing', model_revision='v1.1.1')
File "D:\Anaconda3\envs\anytext\lib\site-packages\modelscope\pipelines\builder.py", line 170, in pipeline
return build_pipeline(cfg, task_name=task)
File "D:\Anaconda3\envs\anytext\lib\site-packages\modelscope\pipelines\builder.py", line 65, in build_pipeline
return build_from_cfg(
File "D:\Anaconda3\envs\anytext\lib\site-packages\modelscope\utils\registry.py", line 215, in build_from_cfg
raise type(e)(f'{obj_cls.name}: {e}')
OSError: AnyTextPipeline: cannot open resource

Ubuntu no Airal Font! (Solved)

Hi,

I install environment using environment.yaml

There is an error loading specific fonts on Ubuntu machines

And I assume code was written for windows machine only?

The cv_anytext_text_generation_editing model would require loading a font by default, the font default path is font/Arial_Unicode.ttf in modelscope 1.10.0

2024-01-02 07:10:36,030 - modelscope - INFO - PyTorch version 2.0.1 Found.
2024-01-02 07:10:36,031 - modelscope - INFO - TensorFlow version 2.13.0 Found.
2024-01-02 07:10:36,031 - modelscope - INFO - Loading ast index from /home/hangyu5/.cache/modelscope/ast_indexer
2024-01-02 07:10:36,062 - modelscope - INFO - Loading done! Current index file version is 1.10.0, with md5 c8dda4735ccce9e1c5d4bb482f4b5560 and a total number of 946 components indexed
2024-01-02 07:10:37,460 - modelscope - INFO - Use user-specified model revision: v1.1.0
2024-01-02 07:10:40,421 - modelscope - WARNING - ('PIPELINES', 'my-anytext-task', 'my-custom-pipeline') not found in ast index file
2024-01-02 07:10:40,421 - modelscope - INFO - initiate model from /home/hangyu5/.cache/modelscope/hub/damo/cv_anytext_text_generation_editing
2024-01-02 07:10:40,421 - modelscope - INFO - initiate model from location /home/hangyu5/.cache/modelscope/hub/damo/cv_anytext_text_generation_editing.
2024-01-02 07:10:40,425 - modelscope - INFO - initialize model from /home/hangyu5/.cache/modelscope/hub/damo/cv_anytext_text_generation_editing
2024-01-02 07:10:40,427 - modelscope - WARNING - ('MODELS', 'my-anytext-task', 'my-custom-model') not found in ast index file
Traceback (most recent call last):
  File "/home/hangyu5/anaconda3/envs/anytext/lib/python3.10/site-packages/modelscope/utils/registry.py", line 210, in build_from_cfg
    return obj_cls._instantiate(**args)
  File "/home/hangyu5/anaconda3/envs/anytext/lib/python3.10/site-packages/modelscope/models/base/base_model.py", line 67, in _instantiate
    return cls(**kwargs)
  File "/home/hangyu5/.cache/modelscope/modelscope_modules/cv_anytext_text_generation_editing/ms_wrapper.py", line 43, in __init__
    self.init_model(**kwargs)
  File "/home/hangyu5/.cache/modelscope/modelscope_modules/cv_anytext_text_generation_editing/ms_wrapper.py", line 218, in init_model
    self.font = ImageFont.truetype(font_path, size=60)
  File "/home/hangyu5/anaconda3/envs/anytext/lib/python3.10/site-packages/PIL/ImageFont.py", line 996, in truetype
    return freetype(font)
  File "/home/hangyu5/anaconda3/envs/anytext/lib/python3.10/site-packages/PIL/ImageFont.py", line 993, in freetype
    return FreeTypeFont(font, size, index, encoding, layout_engine)
  File "/home/hangyu5/anaconda3/envs/anytext/lib/python3.10/site-packages/PIL/ImageFont.py", line 248, in __init__
    self.font = core.getfont(
OSError: cannot open resource

TypeError: function takes exactly 5 arguments (1 given)

Hi,

I tried to launch the project on windows 10 with a 3090, added import os os.environ["KMP_DUPLICATE_LIB_OK"]="TRUE"
and got the followings errors :

(anytext) E:\repo\AnyText>python demo.py
2024-01-08 11:50:28,862 - modelscope - INFO - PyTorch version 2.0.1+cu118 Found.
2024-01-08 11:50:28,866 - modelscope - INFO - TensorFlow version 2.13.0 Found.
2024-01-08 11:50:28,866 - modelscope - INFO - Loading ast index from C:\Users\USER\.cache\modelscope\ast_indexer
2024-01-08 11:50:28,963 - modelscope - INFO - Loading done! Current index file version is 1.10.0, with md5 407792a6ca3bfb6c73e1d4358a891444 and a total number of 946 components indexed
2024-01-08 11:50:34,252 - modelscope - INFO - Use user-specified model revision: v1.1.1
2024-01-08 11:50:38,802 - modelscope - WARNING - ('PIPELINES', 'my-anytext-task', 'anytext-pipeline') not found in ast index file
A matching Triton is not available, some optimizations will not be enabled.
Error caught was: No module named 'triton'
ControlLDM: Running in eps-prediction mode
Setting up MemoryEfficientCrossAttention. Query dim is 320, context_dim is None and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 320, context_dim is 768 and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 320, context_dim is None and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 320, context_dim is 768 and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 640, context_dim is None and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 640, context_dim is 768 and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 640, context_dim is None and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 640, context_dim is 768 and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is None and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is 768 and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is None and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is 768 and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is None and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is 768 and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is None and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is 768 and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is None and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is 768 and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is None and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is 768 and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 640, context_dim is None and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 640, context_dim is 768 and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 640, context_dim is None and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 640, context_dim is 768 and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 640, context_dim is None and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 640, context_dim is 768 and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 320, context_dim is None and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 320, context_dim is 768 and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 320, context_dim is None and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 320, context_dim is 768 and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 320, context_dim is None and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 320, context_dim is 768 and using 8 heads.
DiffusionWrapper has 859.52 M params.
making attention of type 'vanilla-xformers' with 512 in_channels
building MemoryEfficientAttnBlock with 512 in_channels...
Working with z of shape (1, 4, 32, 32) = 4096 dimensions.
making attention of type 'vanilla-xformers' with 512 in_channels
building MemoryEfficientAttnBlock with 512 in_channels...
Setting up MemoryEfficientCrossAttention. Query dim is 320, context_dim is None and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 320, context_dim is 768 and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 320, context_dim is None and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 320, context_dim is 768 and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 640, context_dim is None and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 640, context_dim is 768 and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 640, context_dim is None and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 640, context_dim is 768 and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is None and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is 768 and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is None and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is 768 and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is None and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is 768 and using 8 heads.
Loaded model config from [models_yaml/anytext_sd15.yaml]
Loaded state_dict from [C:\Users\USER\.cache\modelscope\hub\damo\cv_anytext_text_generation_editing\anytext_v1.1.ckpt]
2024-01-08 11:50:58,008 - modelscope - INFO - initiate model from C:\Users\USER\.cache\modelscope\hub\damo\cv_anytext_text_generation_editing\nlp_csanmt_translation_zh2en
2024-01-08 11:50:58,008 - modelscope - INFO - initiate model from location C:\Users\USER\.cache\modelscope\hub\damo\cv_anytext_text_generation_editing\nlp_csanmt_translation_zh2en.
2024-01-08 11:50:58,014 - modelscope - INFO - initialize model from C:\Users\USER\.cache\modelscope\hub\damo\cv_anytext_text_generation_editing\nlp_csanmt_translation_zh2en
{'hidden_size': 1024, 'filter_size': 4096, 'num_heads': 16, 'num_encoder_layers': 24, 'num_decoder_layers': 6, 'attention_dropout': 0.0, 'residual_dropout': 0.0, 'relu_dropout': 0.0, 'layer_preproc': 'layer_norm', 'layer_postproc': 'none', 'shared_embedding_and_softmax_weights': True, 'shared_source_target_embedding': True, 'initializer_scale': 0.1, 'position_info_type': 'absolute', 'max_relative_dis': 16, 'num_semantic_encoder_layers': 4, 'src_vocab_size': 50000, 'trg_vocab_size': 50000, 'seed': 1234, 'beam_size': 4, 'lp_rate': 0.6, 'max_decoded_trg_len': 100, 'device_map': None, 'device': 'cuda'}
2024-01-08 11:50:58,026 - modelscope - WARNING - No val key and type key found in preprocessor domain of configuration.json file.
2024-01-08 11:50:58,027 - modelscope - WARNING - Cannot find available config to build preprocessor at mode inference, current config: {'src_lang': 'zh', 'tgt_lang': 'en', 'src_bpe': {'file': 'bpe.zh'}, 'model_dir': 'C:\\Users\\USER\\.cache\\modelscope\\hub\\damo\\cv_anytext_text_generation_editing\\nlp_csanmt_translation_zh2en'}. trying to build by task and model information.
2024-01-08 11:50:58,027 - modelscope - WARNING - No preprocessor key ('csanmt-translation', 'translation') found in PREPROCESSOR_MAP, skip building preprocessor.
Traceback (most recent call last):
  File "E:\conda\envs\anytext\lib\site-packages\modelscope\utils\registry.py", line 212, in build_from_cfg
    return obj_cls(**args)
  File "E:\conda\envs\anytext\lib\site-packages\modelscope\pipelines\nlp\translation_pipeline.py", line 54, in __init__
    self._src_vocab = dict([
  File "E:\conda\envs\anytext\lib\site-packages\modelscope\pipelines\nlp\translation_pipeline.py", line 54, in <listcomp>
    self._src_vocab = dict([
  File "E:\conda\envs\anytext\lib\encodings\cp1252.py", line 23, in decode
    return codecs.charmap_decode(input,self.errors,decoding_table)[0]
UnicodeDecodeError: 'charmap' codec can't decode byte 0x81 in position 29: character maps to <undefined>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "E:\conda\envs\anytext\lib\site-packages\modelscope\utils\registry.py", line 212, in build_from_cfg
    return obj_cls(**args)
  File "C:\Users\USER\.cache\modelscope\modelscope_modules\cv_anytext_text_generation_editing\ms_wrapper.py", line 336, in __init__
    pipe_model = AnyTextModel(model_dir=model, **kwargs)
  File "C:\Users\USER\.cache\modelscope\modelscope_modules\cv_anytext_text_generation_editing\ms_wrapper.py", line 46, in __init__
    self.init_model(**kwargs)
  File "C:\Users\USER\.cache\modelscope\modelscope_modules\cv_anytext_text_generation_editing\ms_wrapper.py", line 240, in init_model
    self.trans_pipe = pipeline(task=Tasks.translation, model=os.path.join(self.model_dir, 'nlp_csanmt_translation_zh2en'))
  File "E:\conda\envs\anytext\lib\site-packages\modelscope\pipelines\builder.py", line 170, in pipeline
    return build_pipeline(cfg, task_name=task)
  File "E:\conda\envs\anytext\lib\site-packages\modelscope\pipelines\builder.py", line 65, in build_pipeline
    return build_from_cfg(
  File "E:\conda\envs\anytext\lib\site-packages\modelscope\utils\registry.py", line 215, in build_from_cfg
    raise type(e)(f'{obj_cls.__name__}: {e}')
TypeError: function takes exactly 5 arguments (1 given)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "E:\repo\AnyText\demo.py", line 53, in <module>
    inference = pipeline('my-anytext-task', model='damo/cv_anytext_text_generation_editing', model_revision='v1.1.1', use_fp16=not args.use_fp32, use_translator=not args.no_translator, font_path=args.font_path)
  File "E:\conda\envs\anytext\lib\site-packages\modelscope\pipelines\builder.py", line 170, in pipeline
    return build_pipeline(cfg, task_name=task)
  File "E:\conda\envs\anytext\lib\site-packages\modelscope\pipelines\builder.py", line 65, in build_pipeline
    return build_from_cfg(
  File "E:\conda\envs\anytext\lib\site-packages\modelscope\utils\registry.py", line 215, in build_from_cfg
    raise type(e)(f'{obj_cls.__name__}: {e}')
TypeError: AnyTextPipeline: function takes exactly 5 arguments (1 given)

(anytext) E:\repo\AnyText>

Not found in ast index file

对这个项目很感兴趣，但我是一个靠GPT4来指导我部署anytext的新新手，请问我这是碰到什么问题了？（GPT貌似都有些束手无策了）

(anytext) C:\Windows\System32\Anytext>python inference.py
2024-01-03 14:23:34,487 - modelscope - INFO - PyTorch version 2.1.2+cu118 Found.
2024-01-03 14:23:34,489 - modelscope - INFO - TensorFlow version 2.13.0 Found.
2024-01-03 14:23:34,489 - modelscope - INFO - Loading ast index from C:\Users\HP.cache\modelscope\ast_indexer
2024-01-03 14:23:34,572 - modelscope - INFO - Loading done! Current index file version is 1.10.0, with md5 026f7c772a5d4e347142d024af000099 and a total number of 946 components indexed
2024-01-03 14:23:35,808 - modelscope - INFO - Use user-specified model revision: v1.1.0
2024-01-03 14:23:38,550 - modelscope - WARNING - ('PIPELINES', 'my-anytext-task', 'my-custom-pipeline') not found in ast index file
2024-01-03 14:23:38,551 - modelscope - INFO - initiate model from C:\Users\HP.cache\modelscope\hub\damo\cv_anytext_text_generation_editing
2024-01-03 14:23:38,552 - modelscope - INFO - initiate model from location C:\Users\HP.cache\modelscope\hub\damo\cv_anytext_text_generation_editing.
2024-01-03 14:23:38,555 - modelscope - INFO - initialize model from C:\Users\HP.cache\modelscope\hub\damo\cv_anytext_text_generation_editing
2024-01-03 14:23:38,559 - modelscope - WARNING - ('MODELS', 'my-anytext-task', 'my-custom-model') not found in ast index file
A matching Triton is not available, some optimizations will not be enabled.
Error caught was: No module named 'triton'
OMP: Error #15: Initializing libiomp5md.dll, but found libiomp5md.dll already initialized.
OMP: Hint This means that multiple copies of the OpenMP runtime have been linked into the program. That is dangerous, since it can degrade performance or cause incorrect results. The best thing to do is to ensure that only a single OpenMP runtime is linked into the process, e.g. by avoiding static linking of the OpenMP runtime in any library. As an unsafe, unsupported, undocumented workaround you can set the environment variable KMP_DUPLICATE_LIB_OK=TRUE to allow the program to continue to execute, but that may cause crashes or silently produce incorrect results. For more information, please see http://www.intel.com/software/products/support/.

(anytext) C:\Windows\System32\Anytext>

It is recommended to add font selection

I have tried many pictures, but some fonts cannot be replaced. It is recommended to increase font selection, so that different fonts can be recognized, which can greatly increase the robustness of the system

anytext_v1.1.ckpt integrity check failed

I tried installing AnyText according to the instructions, but it failed at the checkpoint downloading step. The error message is copied as follows. How do I fix this?

python demo.py
2024-01-03 14:58:58,887 - modelscope - INFO - PyTorch version 2.0.1 Found.
2024-01-03 14:58:58,890 - modelscope - INFO - TensorFlow version 2.13.0 Found.
2024-01-03 14:58:58,890 - modelscope - INFO - Loading ast index from C:\Users\username.cache\modelscope\ast_indexer
2024-01-03 14:58:58,988 - modelscope - INFO - Loading done! Current index file version is 1.10.0, with md5 08d3d34944933ce0fe70855f540131f5 and a total number of 946 components indexed
2024-01-03 14:59:02,484 - modelscope - INFO - Use user-specified model revision: v1.1.0
Downloading: 100%|████████████████████████████████████████████████████████████▉| 5.34G/5.34G [34:17<00:00, 2.78MB/s]
2024-01-03 15:33:32,806 - modelscope - ERROR - File C:\Users\username.cache\modelscope\hub\temp\tmppr3kigdn\anytext_v1.1.ckpt integrity check failed, the download may be incomplete, please try again.
Traceback (most recent call last):
File "C:\Users\username\Documents\ASTRI\repo\fake_detect\AnyText\demo.py", line 21, in
inference = pipeline('my-anytext-task', model='damo/cv_anytext_text_generation_editing', model_revision='v1.1.0')
File "C:\Users\username\anaconda3\envs\anytext\lib\site-packages\modelscope\pipelines\builder.py", line 115, in pipeline
model = normalize_model_input(
File "C:\Users\username\anaconda3\envs\anytext\lib\site-packages\modelscope\pipelines\builder.py", line 36, in normalize_model_input
model = snapshot_download(
File "C:\Users\username\anaconda3\envs\anytext\lib\site-packages\modelscope\hub\snapshot_download.py", line 159, in snapshot_download
file_integrity_validation(temp_file, model_file[FILE_HASH])
File "C:\Users\username\anaconda3\envs\anytext\lib\site-packages\modelscope\hub\utils\utils.py", line 94, in file_integrity_validation
raise FileIntegrityError(msg)
modelscope.hub.errors.FileIntegrityError: File C:\Users\username.cache\modelscope\hub\temp\tmppr3kigdn\anytext_v1.1.ckpt integrity check failed, the download may be incomplete, please try again.

Can not run

Traceback (most recent call last):
File "demo.py", line 33, in
class ScriptLoader:
File "demo.py", line 45, in ScriptLoader
def get_scripts(path: str, file_type: str) -> list[tuple[str, str]]:
TypeError: 'type' object is not subscriptable

I got this error evnetually when installed torch tensorflow and whole universe this project need....

No preprocessor key ('my-custom-model', 'my-anytext-task') found in PREPROCESSOR_MAP, skip building preprocessor

工作很棒。
但是使用给的代码完全不能复现效果。怀疑是这个warning的提示的原因：No preprocessor key ('my-custom-model', 'my-anytext-task') found in PREPROCESSOR_MAP, skip building preprocessor
也就是说my-anytext-task 这个task的预处理流程没有定义。是因为modelscope的版本问题吗？或者其他原因？
谢谢！

Windows 11 error Torch not compiled with CUDA enabled

(anytext) PS I:\GitHub\AnyText> python inference.py
2024-01-03 21:32:25,086 - modelscope - INFO - PyTorch version 2.0.1 Found.
2024-01-03 21:32:25,093 - modelscope - INFO - TensorFlow version 2.13.0 Found.
2024-01-03 21:32:25,093 - modelscope - INFO - Loading ast index from C:\Users\wyj35.cache\modelscope\ast_indexer
2024-01-03 21:32:25,476 - modelscope - INFO - Updating the files for the changes of local files, first time updating will take longer time! Please wait till updating done!
2024-01-03 21:32:25,497 - modelscope - INFO - AST-Scanning the path "C:\Users\wyj35\miniconda3\envs\anytext\Lib\site-packages\modelscope" with the following sub folders ['models', 'metrics', 'pipelines', 'preprocessors', 'trainers', 'msdatasets', 'exporters']
2024-01-03 21:32:48,806 - modelscope - INFO - Scanning done! A number of 946 components indexed or updated! Time consumed 23.30835795402527s
2024-01-03 21:32:49,189 - modelscope - INFO - Loading done! Current index file version is 1.10.0, with md5 827cc68d432b6aa086f40fca407a8421 and a total number of 946 components indexed
2024-01-03 21:32:52,404 - modelscope - INFO - Use user-specified model revision: v1.1.0
2024-01-03 21:32:59,594 - modelscope - WARNING - ('PIPELINES', 'my-anytext-task', 'my-custom-pipeline') not found in ast index file
2024-01-03 21:32:59,595 - modelscope - INFO - initiate model from C:\Users\wyj35.cache\modelscope\hub\damo\cv_anytext_text_generation_editing
2024-01-03 21:32:59,595 - modelscope - INFO - initiate model from location C:\Users\wyj35.cache\modelscope\hub\damo\cv_anytext_text_generation_editing.
2024-01-03 21:32:59,601 - modelscope - INFO - initialize model from C:\Users\wyj35.cache\modelscope\hub\damo\cv_anytext_text_generation_editing
2024-01-03 21:32:59,609 - modelscope - WARNING - ('MODELS', 'my-anytext-task', 'my-custom-model') not found in ast index file
WARNING[XFORMERS]: xFormers can't load C++/CUDA extensions. xFormers was built for:
PyTorch 2.0.1+cu118 with CUDA 1108 (you have 2.0.1+cpu)
Python 3.10.11 (you have 3.10.6)
Please reinstall xformers (see https://github.com/facebookresearch/xformers#installing-xformers)
Memory-efficient attention, SwiGLU, sparse and more won't be available.
Set XFORMERS_MORE_DETAILS=1 for more details
ControlLDM: Running in eps-prediction mode
Setting up MemoryEfficientCrossAttention. Query dim is 320, context_dim is None and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 320, context_dim is 768 and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 320, context_dim is None and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 320, context_dim is 768 and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 640, context_dim is None and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 640, context_dim is 768 and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 640, context_dim is None and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 640, context_dim is 768 and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is None and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is 768 and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is None and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is 768 and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is None and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is 768 and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is None and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is 768 and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is None and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is 768 and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is None and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is 768 and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 640, context_dim is None and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 640, context_dim is 768 and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 640, context_dim is None and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 640, context_dim is 768 and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 640, context_dim is None and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 640, context_dim is 768 and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 320, context_dim is None and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 320, context_dim is 768 and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 320, context_dim is None and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 320, context_dim is 768 and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 320, context_dim is None and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 320, context_dim is 768 and using 8 heads.
DiffusionWrapper has 859.52 M params.
making attention of type 'vanilla-xformers' with 512 in_channels
building MemoryEfficientAttnBlock with 512 in_channels...
Working with z of shape (1, 4, 32, 32) = 4096 dimensions.
making attention of type 'vanilla-xformers' with 512 in_channels
building MemoryEfficientAttnBlock with 512 in_channels...
Setting up MemoryEfficientCrossAttention. Query dim is 320, context_dim is None and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 320, context_dim is 768 and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 320, context_dim is None and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 320, context_dim is 768 and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 640, context_dim is None and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 640, context_dim is 768 and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 640, context_dim is None and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 640, context_dim is 768 and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is None and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is 768 and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is None and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is 768 and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is None and using 8 heads.
Setting up MemoryEfficientCrossAttention. Query dim is 1280, context_dim is 768 and using 8 heads.
Loaded model config from [models_yaml/anytext_sd15.yaml]
Traceback (most recent call last):
File "C:\Users\wyj35\miniconda3\envs\anytext\lib\site-packages\modelscope\utils\registry.py", line 210, in build_from_cfg
return obj_cls._instantiate(**args)
File "C:\Users\wyj35\miniconda3\envs\anytext\lib\site-packages\modelscope\models\base\base_model.py", line 67, in _instantiate
return cls(**kwargs)
File "C:\Users\wyj35.cache\modelscope\modelscope_modules\cv_anytext_text_generation_editing\ms_wrapper.py", line 43, in init
self.init_model(**kwargs)
File "C:\Users\wyj35.cache\modelscope\modelscope_modules\cv_anytext_text_generation_editing\ms_wrapper.py", line 222, in init_model
self.model = create_model(cfg_path, cond_stage_path=clip_path).cuda().eval()
File "C:\Users\wyj35\miniconda3\envs\anytext\lib\site-packages\pytorch_lightning\core\mixins\device_dtype_mixin.py", line 126, in cuda
return super().cuda(device=device)
File "C:\Users\wyj35\miniconda3\envs\anytext\lib\site-packages\torch\nn\modules\module.py", line 905, in cuda
return self._apply(lambda t: t.cuda(device))
File "C:\Users\wyj35\miniconda3\envs\anytext\lib\site-packages\torch\nn\modules\module.py", line 797, in _apply
module._apply(fn)
File "C:\Users\wyj35\miniconda3\envs\anytext\lib\site-packages\torch\nn\modules\module.py", line 797, in _apply
module._apply(fn)
File "C:\Users\wyj35\miniconda3\envs\anytext\lib\site-packages\torch\nn\modules\module.py", line 797, in _apply
module._apply(fn)
[Previous line repeated 1 more time]
File "C:\Users\wyj35\miniconda3\envs\anytext\lib\site-packages\torch\nn\modules\module.py", line 820, in _apply
param_applied = fn(param)
File "C:\Users\wyj35\miniconda3\envs\anytext\lib\site-packages\torch\nn\modules\module.py", line 905, in
return self.apply(lambda t: t.cuda(device))
File "C:\Users\wyj35\miniconda3\envs\anytext\lib\site-packages\torch\cuda_init.py", line 239, in _lazy_init
raise AssertionError("Torch not compiled with CUDA enabled")
AssertionError: Torch not compiled with CUDA enabled

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "C:\Users\wyj35\miniconda3\envs\anytext\lib\site-packages\modelscope\utils\registry.py", line 212, in build_from_cfg
return obj_cls(**args)
File "C:\Users\wyj35.cache\modelscope\modelscope_modules\cv_anytext_text_generation_editing\ms_wrapper.py", line 320, in init
super().init(model=model, auto_collate=False)
File "C:\Users\wyj35\miniconda3\envs\anytext\lib\site-packages\modelscope\pipelines\base.py", line 99, in init
self.model = self.initiate_single_model(model)
File "C:\Users\wyj35\miniconda3\envs\anytext\lib\site-packages\modelscope\pipelines\base.py", line 53, in initiate_single_model
return Model.from_pretrained(
File "C:\Users\wyj35\miniconda3\envs\anytext\lib\site-packages\modelscope\models\base\base_model.py", line 183, in from_pretrained
model = build_model(model_cfg, task_name=task_name)
File "C:\Users\wyj35\miniconda3\envs\anytext\lib\site-packages\modelscope\models\builder.py", line 35, in build_model
model = build_from_cfg(
File "C:\Users\wyj35\miniconda3\envs\anytext\lib\site-packages\modelscope\utils\registry.py", line 215, in build_from_cfg
raise type(e)(f'{obj_cls.name}: {e}')
AssertionError: MyCustomModel: Torch not compiled with CUDA enabled

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "I:\GitHub\AnyText\inference.py", line 3, in
pipe = pipeline('my-anytext-task', model='damo/cv_anytext_text_generation_editing', model_revision='v1.1.0')
File "C:\Users\wyj35\miniconda3\envs\anytext\lib\site-packages\modelscope\pipelines\builder.py", line 170, in pipeline
return build_pipeline(cfg, task_name=task)
File "C:\Users\wyj35\miniconda3\envs\anytext\lib\site-packages\modelscope\pipelines\builder.py", line 65, in build_pipeline
return build_from_cfg(
File "C:\Users\wyj35\miniconda3\envs\anytext\lib\site-packages\modelscope\utils\registry.py", line 215, in build_from_cfg
raise type(e)(f'{obj_cls.name}: {e}')
AssertionError: MyCustomPipeline: MyCustomModel: Torch not compiled with CUDA enabled

run error : MyCustomModel: cannot open resource

python demo.py
2023-12-27 13:25:47,978 - modelscope - INFO - PyTorch version 2.1.2 Found.
2023-12-27 13:25:47,981 - modelscope - INFO - TensorFlow version 2.15.0.post1 Found.
2023-12-27 13:25:47,981 - modelscope - INFO - Loading ast index from /share/model/cv_anytext_text_generation_editing/ast_indexer
2023-12-27 13:25:48,094 - modelscope - INFO - Loading done! Current index file version is 1.10.0, with md5 099672c06c5dce8e4240f79ebe0fd960 and a total number of 946 components indexed
2023-12-27 13:25:52,146 - modelscope - INFO - Use user-specified model revision: v1.1.0
Downloading: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████| 56.0/56.0 [00:00<00:00, 426kB/s]
Downloading: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████| 1.06k/1.06k [00:00<00:00, 7.98MB/s]
Downloading: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████▉| 5.34G/5.34G [02:42<00:00, 35.3MB/s]
Downloading: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████| 345k/345k [00:00<00:00, 2.22MB/s]
Downloading: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████| 373k/373k [00:00<00:00, 2.40MB/s]
Downloading: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████| 69.0/69.0 [00:00<00:00, 511kB/s]
Downloading: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████▉| 7.34G/7.34G [03:31<00:00, 37.2MB/s]
Downloading: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████| 85.8k/85.8k [00:00<00:00, 823kB/s]
Downloading: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████| 10.0M/10.0M [00:00<00:00, 17.6MB/s]
Downloading: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████| 4.41k/4.41k [00:00<00:00, 289kB/s]
Downloading: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████| 2.07k/2.07k [00:00<00:00, 19.0MB/s]
Downloading: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████| 144/144 [00:00<00:00, 1.49MB/s]
Downloading: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████| 62.5k/62.5k [00:00<00:00, 838kB/s]
Downloading: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████| 62.9k/62.9k [00:00<00:00, 935kB/s]
Downloading: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████| 334k/334k [00:00<00:00, 2.09MB/s]
Downloading: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████| 512k/512k [00:00<00:00, 3.56MB/s]
Downloading: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████| 17.4k/17.4k [00:00<00:00, 617kB/s]
Downloading: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████| 316/316 [00:00<00:00, 2.35MB/s]
Downloading: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████▉| 1.59G/1.59G [00:50<00:00, 34.2MB/s]
Downloading: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████| 5.39k/5.39k [00:00<00:00, 419kB/s]
Downloading: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████| 7.76k/7.76k [00:00<00:00, 32.1MB/s]
Downloading: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████| 9.26k/9.26k [00:00<00:00, 256kB/s]
Downloading: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████| 546k/546k [00:00<00:00, 1.63MB/s]
Downloading: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████| 60.7k/60.7k [00:00<00:00, 677kB/s]
Downloading: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████| 389/389 [00:00<00:00, 3.48MB/s]
Downloading: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████| 327k/327k [00:00<00:00, 1.53MB/s]
Downloading: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████| 2.12M/2.12M [00:00<00:00, 2.69MB/s]
Downloading: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████| 905/905 [00:00<00:00, 5.90MB/s]
Downloading: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████| 152k/152k [00:00<00:00, 1.56MB/s]
Downloading: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████| 138k/138k [00:00<00:00, 1.48MB/s]
Downloading: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████| 320k/320k [00:00<00:00, 1.85MB/s]
Downloading: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████| 939k/939k [00:00<00:00, 2.64MB/s]
2023-12-27 13:34:17,636 - modelscope - WARNING - ('PIPELINES', 'my-anytext-task', 'my-custom-pipeline') not found in ast index file
2023-12-27 13:34:17,636 - modelscope - INFO - initiate model from /share/model/cv_anytext_text_generation_editing/damo/cv_anytext_text_generation_editing
2023-12-27 13:34:17,641 - modelscope - INFO - initiate model from location /share/model/cv_anytext_text_generation_editing/damo/cv_anytext_text_generation_editing.
2023-12-27 13:34:17,643 - modelscope - INFO - initialize model from /share/model/cv_anytext_text_generation_editing/damo/cv_anytext_text_generation_editing
2023-12-27 13:34:17,658 - modelscope - WARNING - ('MODELS', 'my-anytext-task', 'my-custom-model') not found in ast index file
Traceback (most recent call last):
File "/opt/miniconda3/envs/anytext/lib/python3.10/site-packages/modelscope/utils/registry.py", line 210, in build_from_cfg
return obj_cls._instantiate(**args)
File "/opt/miniconda3/envs/anytext/lib/python3.10/site-packages/modelscope/models/base/base_model.py", line 67, in _instantiate
return cls(**kwargs)
File "/root/.cache/modelscope/modelscope_modules/cv_anytext_text_generation_editing/ms_wrapper.py", line 43, in init
self.init_model(**kwargs)
File "/root/.cache/modelscope/modelscope_modules/cv_anytext_text_generation_editing/ms_wrapper.py", line 218, in init_model
self.font = ImageFont.truetype(font_path, size=60)
File "/opt/miniconda3/envs/anytext/lib/python3.10/site-packages/PIL/ImageFont.py", line 791, in truetype
return freetype(font)
File "/opt/miniconda3/envs/anytext/lib/python3.10/site-packages/PIL/ImageFont.py", line 788, in freetype
return FreeTypeFont(font, size, index, encoding, layout_engine)
File "/opt/miniconda3/envs/anytext/lib/python3.10/site-packages/PIL/ImageFont.py", line 226, in init
self.font = core.getfont(
OSError: cannot open resource

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/opt/miniconda3/envs/anytext/lib/python3.10/site-packages/modelscope/utils/registry.py", line 212, in build_from_cfg
return obj_cls(**args)
File "/root/.cache/modelscope/modelscope_modules/cv_anytext_text_generation_editing/ms_wrapper.py", line 320, in init
super().init(model=model, auto_collate=False)
File "/opt/miniconda3/envs/anytext/lib/python3.10/site-packages/modelscope/pipelines/base.py", line 99, in init
self.model = self.initiate_single_model(model)
File "/opt/miniconda3/envs/anytext/lib/python3.10/site-packages/modelscope/pipelines/base.py", line 53, in initiate_single_model
return Model.from_pretrained(
File "/opt/miniconda3/envs/anytext/lib/python3.10/site-packages/modelscope/models/base/base_model.py", line 183, in from_pretrained
model = build_model(model_cfg, task_name=task_name)
File "/opt/miniconda3/envs/anytext/lib/python3.10/site-packages/modelscope/models/builder.py", line 35, in build_model
model = build_from_cfg(
File "/opt/miniconda3/envs/anytext/lib/python3.10/site-packages/modelscope/utils/registry.py", line 215, in build_from_cfg
raise type(e)(f'{obj_cls.name}: {e}')
OSError: MyCustomModel: cannot open resource

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/share/ai/AnyText-main/demo.py", line 20, in
inference = pipeline('my-anytext-task', model='damo/cv_anytext_text_generation_editing', model_revision='v1.1.0')
File "/opt/miniconda3/envs/anytext/lib/python3.10/site-packages/modelscope/pipelines/builder.py", line 170, in pipeline
return build_pipeline(cfg, task_name=task)
File "/opt/miniconda3/envs/anytext/lib/python3.10/site-packages/modelscope/pipelines/builder.py", line 65, in build_pipeline
return build_from_cfg(
File "/opt/miniconda3/envs/anytext/lib/python3.10/site-packages/modelscope/utils/registry.py", line 215, in build_from_cfg
raise type(e)(f'{obj_cls.name}: {e}')
OSError: MyCustomPipeline: MyCustomModel: cannot open resource

The lib are toooooooo old

Gradio now is 4.30
You still using 3.50

is that possible have some upgrade ? I thin you can not use 3.50 for the whole life, you have to upgrade it someday, really suggest do not using toooooo old lib

Anytext是不是不能在Windows11上运行呢？只能Linux？

家人们，谁懂啊？

显卡4090，cuda12.3，死活跑不起来…

由于网络原因,下载anytext_v1.1.ckpt失败

由于隔离网络环境连不上modelscope,我自行下载了官方提供的模型文件夹/cv_anytext_text_generation_editing/, 并放在了~/.cache/modelscope/hub/damo/目录下,但是在运行demo.py时依然会自动执行下载导致报错,请问如何绕过自动下载?

Availability on huggingface?

Can you upload the model on huggingface?

is possible run AnyText on Mac M2

try to change modelscope , add device="mps" , and disable yaml cudatoolkit , not working . Can you give some suggestion ?
thanks

How much GPU memory to run inference.py?

I tried to run inference, but failed 'CUDA out of memory', I'm using 8GB RTX 3080 TI, does it work?

ImportError: libcupti.so.11.7: cannot open shared object file: No such file or directory

File "/home/ian/projs/AnyText/inference.py", line 1, in
from modelscope.pipelines import pipeline
File "/media/ian/extras/condaenvs/anytext/lib/python3.10/site-packages/modelscope/init.py", line 4, in
from modelscope.utils.import_utils import LazyImportModule
File "/media/ian/extras/condaenvs/anytext/lib/python3.10/site-packages/modelscope/utils/init.py", line 1, in
from .hub import create_model_if_not_exist, read_config
File "/media/ian/extras/condaenvs/anytext/lib/python3.10/site-packages/modelscope/utils/hub.py", line 10, in
from modelscope.hub.file_download import model_file_download
File "/media/ian/extras/condaenvs/anytext/lib/python3.10/site-packages/modelscope/hub/file_download.py", line 18, in
from modelscope.hub.api import HubApi, ModelScopeConfig
File "/media/ian/extras/condaenvs/anytext/lib/python3.10/site-packages/modelscope/hub/api.py", line 36, in
from modelscope.hub.errors import (InvalidParameter, NotExistError,
File "/media/ian/extras/condaenvs/anytext/lib/python3.10/site-packages/modelscope/hub/errors.py", line 11, in
logger = get_logger()
File "/media/ian/extras/condaenvs/anytext/lib/python3.10/site-packages/modelscope/utils/logger.py", line 43, in get_logger
from modelscope.utils.torch_utils import is_dist, is_master
File "/media/ian/extras/condaenvs/anytext/lib/python3.10/site-packages/modelscope/utils/torch_utils.py", line 13, in
import torch
File "/media/ian/extras/condaenvs/anytext/lib/python3.10/site-packages/torch/init.py", line 229, in
from torch._C import * # noqa: F403
ImportError: libcupti.so.11.7: cannot open shared object file: No such file or directory

my torch version:

torch 2.0.1
torchdiffeq 0.2.3
torchmetrics 0.11.4
torchvision 0.16.0
tornado 6.2
tqdm 4.66.1

How to control

How do I control the color, font, and size of the generated letters?

When can we expect training script to be released for AnyText?

Issue Regarding the Installation of Arial Unicode Font

Dear AnyText Team,

I hope this message finds you well. I am reaching out to express a concern I encountered while setting up the AnyText environment as per the instructions provided in the README.md file.

The issue pertains to the installation of the Arial Unicode font. The current instructions suggest moving an existing Arial Unicode font file to the specified directory within the AnyText project. However, it appears that the font file must be procured independently by the user, which might not be straightforward for everyone.

Given that Arial Unicode MS is a proprietary font, users might face difficulties in legally obtaining it, especially if they do not have access to it through a licensed copy of Microsoft Office or other authorised sources. This could potentially hinder the accessibility of AnyText for some users who are eager to explore its capabilities.

Might I suggest that we include additional guidance on how users can legally acquire the Arial Unicode font or recommend alternative fonts that are freely available and compatible with AnyText? This would greatly assist users in setting up the environment without any legal or technical impediments.

I appreciate your attention to this matter and look forward to your response. Thank you for your dedication to improving the user experience of AnyText.

Best regards,
yihong1120

Killed

ControlLDM: Running in eps-prediction mode
DiffusionWrapper has 859.52 M params.
making attention of type 'vanilla' with 512 in_channels
Killed

it just killed when run python demo.py why

推理跑不起来，这是什么原因呢？

(anytext) C:\Users\sunny\Documents\AnyText>python inference.py
2024-01-04 18:24:07,722 - modelscope - INFO - PyTorch version 2.1.2+cu121 Found.
2024-01-04 18:24:07,724 - modelscope - INFO - TensorFlow version 2.13.0 Found.
2024-01-04 18:24:07,724 - modelscope - INFO - Loading ast index from C:\Users\sunny.cache\modelscope\ast_indexer
2024-01-04 18:24:07,772 - modelscope - INFO - Loading done! Current index file version is 1.10.0, with md5 25145d097e3652b81ca7902ed6ed4218 and a total number of 946 components indexed
2024-01-04 18:24:08,928 - modelscope - INFO - Use user-specified model revision: v1.1.0
2024-01-04 18:24:11,285 - modelscope - WARNING - ('PIPELINES', 'my-anytext-task', 'my-custom-pipeline') not found in ast index file
2024-01-04 18:24:11,286 - modelscope - INFO - initiate model from C:\Users\sunny.cache\modelscope\hub\damo\cv_anytext_text_generation_editing
2024-01-04 18:24:11,286 - modelscope - INFO - initiate model from location C:\Users\sunny.cache\modelscope\hub\damo\cv_anytext_text_generation_editing.
2024-01-04 18:24:11,287 - modelscope - INFO - initialize model from C:\Users\sunny.cache\modelscope\hub\damo\cv_anytext_text_generation_editing
2024-01-04 18:24:11,289 - modelscope - WARNING - ('MODELS', 'my-anytext-task', 'my-custom-model') not found in ast index file
A matching Triton is not available, some optimizations will not be enabled.
Error caught was: No module named 'triton'
OMP: Error #15: Initializing libiomp5md.dll, but found libiomp5md.dll already initialized.
OMP: Hint This means that multiple copies of the OpenMP runtime have been linked into the program. That is dangerous, since it can degrade performance or cause incorrect results. The best thing to do is to ensure that only a single OpenMP runtime is linked into the process, e.g. by avoiding static linking of the OpenMP runtime in any library. As an unsafe, unsupported, undocumented workaround you can set the environment variable KMP_DUPLICATE_LIB_OK=TRUE to allow the program to continue to execute, but that may cause crashes or silently produce incorrect results. For more information, please see http://www.intel.com/software/products/support/.

Non-OK-status

Global seed set to 40173333
Building prefix dict from the default dictionary ...
Loading model from cache /tmp/jieba.cache
Loading model cost 0.858 seconds.
Prefix dict has been built successfully.
WARNING:tensorflow:From /root/anaconda3/envs/anytext/lib/python3.10/site-packages/modelscope/utils/device.py:60: is_gpu_available (from tensorflow.python.framework.test_util) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.config.list_physical_devices('GPU') instead.
2024-01-05 14:20:40.148754: F ./tensorflow/core/kernels/reduction_gpu_kernels.cu.h:660] Non-OK-status: GpuLaunchKernel(BlockReduceKernel<IN_T, OUT_T, num_threads, Op>, num_blocks, num_threads, 0, cu_stream, in, out, in_size, op, init) status: INTERNAL: cudaGetErrorString symbol not found.
Aborted

可以打开页面，点击run，报错退出。
求解～

Error reported during reasoning——“NotImplementedError: No operator found for `memory_efficient_attention_forward` with inputs:”

Translate: 一只浣熊站在黑板前，上面写着 * --> A raccoon stands in front of the blackboard with the words *
Traceback (most recent call last):
File "D:\python\test\ai\AnyText\venv\Lib\site-packages\gradio\queueing.py", line 407, in call_prediction
output = await route_utils.call_process_api(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\python\test\ai\AnyText\venv\Lib\site-packages\gradio\route_utils.py", line 226, in call_process_api
output = await app.get_blocks().process_api(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\python\test\ai\AnyText\venv\Lib\site-packages\gradio\blocks.py", line 1550, in process_api
result = await self.call_function(
^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\python\test\ai\AnyText\venv\Lib\site-packages\gradio\blocks.py", line 1185, in call_function
prediction = await anyio.to_thread.run_sync(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\python\test\ai\AnyText\venv\Lib\site-packages\anyio\to_thread.py", line 33, in run_sync
return await get_asynclib().run_sync_in_worker_thread(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\python\test\ai\AnyText\venv\Lib\site-packages\anyio_backends_asyncio.py", line 877, in run_sync_in_worker_thread
return await future
^^^^^^^^^^^^
File "D:\python\test\ai\AnyText\venv\Lib\site-packages\anyio_backends_asyncio.py", line 807, in run
result = context.run(func, *args)
^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\python\test\ai\AnyText\venv\Lib\site-packages\gradio\utils.py", line 661, in wrapper
response = f(*args, **kwargs)
^^^^^^^^^^^^^^^^^^
File "D:\python\test\ai\AnyText\demo.py", line 169, in process
results, rtn_code, rtn_warning, debug_info = inference(input_data, mode=mode, **params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\python\test\ai\AnyText\venv\Lib\site-packages\modelscope\pipelines\base.py", line 219, in call
output = self._process_single(input, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\python\test\ai\AnyText\venv\Lib\site-packages\modelscope\pipelines\base.py", line 254, in _process_single
out = self.forward(out, **forward_params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\chenyixuan.cache\modelscope\modelscope_modules\cv_anytext_text_generation_editing\ms_wrapper.py", line 339, in forward
return super().forward(inputs, **forward_params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\python\test\ai\AnyText\venv\Lib\site-packages\modelscope\pipelines\base.py", line 397, in forward
return self.model(inputs, **forward_params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\python\test\ai\AnyText\venv\Lib\site-packages\modelscope\models\base\base_torch_model.py", line 36, in call
return self.postprocess(self.forward(*args, **kwargs))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\chenyixuan.cache\modelscope\modelscope_modules\cv_anytext_text_generation_editing\ms_wrapper.py", line 176, in forward
encoder_posterior = self.model.encode_first_stage(masked_img[None, ...])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\python\test\ai\AnyText\venv\Lib\site-packages\torch\utils_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "D:\python\test\ai\AnyText\ldm\models\diffusion\ddpm.py", line 870, in encode_first_stage
return self.first_stage_model.encode(x)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\python\test\ai\AnyText\ldm\models\autoencoder.py", line 83, in encode
h = self.encoder(x)
^^^^^^^^^^^^^^^
File "D:\python\test\ai\AnyText\venv\Lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\python\test\ai\AnyText\venv\Lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\python\test\ai\AnyText\ldm\modules\diffusionmodules\model.py", line 536, in forward
h = self.mid.attn_1(h)
^^^^^^^^^^^^^^^^^^
File "D:\python\test\ai\AnyText\venv\Lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\python\test\ai\AnyText\venv\Lib\site-packages\torch\nn\modules\module.py", line 1527, in call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\python\test\ai\AnyText\ldm\modules\diffusionmodules\model.py", line 258, in forward
out = xformers.ops.memory_efficient_attention(q, k, v, attn_bias=None, op=self.attention_op)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\python\test\ai\AnyText\venv\Lib\site-packages\xformers\ops\fmha_init.py", line 192, in memory_efficient_attention
return memory_efficient_attention(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\python\test\ai\AnyText\venv\Lib\site-packages\xformers\ops\fmha_init.py", line 290, in _memory_efficient_attention
return memory_efficient_attention_forward(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\python\test\ai\AnyText\venv\Lib\site-packages\xformers\ops\fmha_init.py", line 306, in _memory_efficient_attention_forward
op = _dispatch_fw(inp)
^^^^^^^^^^^^^^^^^
File "D:\python\test\ai\AnyText\venv\Lib\site-packages\xformers\ops\fmha\dispatch.py", line 94, in _dispatch_fw
return _run_priority_list(
^^^^^^^^^^^^^^^^^^^
File "D:\python\test\ai\AnyText\venv\Lib\site-packages\xformers\ops\fmha\dispatch.py", line 69, in _run_priority_list
raise NotImplementedError(msg)
NotImplementedError: No operator found for memory_efficient_attention_forward with inputs:

Result wrong

Cannot generate Cihnese

Not supporting Arabic

Why Arabic is not supported? It should be multilingual no?