xming521/weclone

使用微信聊天记录微调大语言模型，并绑定到微信机器人，实现自己的数字克隆。 LLM/大语言模型/大模型/微信聊天记录/微信聊天机器人/LoRA

👋 欢迎

主要技术栈

• 😄 使用语言：Python(熟练)/TypeScript(了解)/Scala(了解)
• 😃大数据及分布式（熟悉）：Hadoop/Spark/HBase
• 😆AI模型训练及推理（熟悉）：PyTorch/TensorRT/Triton
• 😝前后端及中间件（了解）：Vue/Nuxt/FastApi/Flask/MySQL/SQLite
• 😊CI/CD及其他：Git/Github Actions/Azure DevOps/Linux/Docker
• 🥰云服务: 阿里云/腾讯云/Azure
• 🤖AI生产力：GitHub Copilot/ChatGPT/New Bing

Languages and Tools:

weclone's Issues

transformers 版本问题

transformers 4.38.1 会报这个错
ImportError: cannot import name 'MixtralBLockSparseTop2MLP' from 'transformers.models.mixtral.modeling_mixtral'

transformers 4.40.1 又会报这个错
ttributeError: module 'ml_dtypes' has no attribute 'float8_e4m3b11'. Did you mean: 'float8_e4m3fn'?
Traceback (most recent call last):
File "Z:\python program\WeClone\src\train_sft.py", line 1, in
from llmtuner import run_exp
File "Z:\python program\WeClone\venv\lib\site-packages\llmtuner_init_.py", line 6, in
from .train import export_model, run_exp
File "Z:\python program\WeClone\venv\lib\site-packages\llmtuner\train_init_.py", line 1, in
from .tuner import export_model, run_exp
File "Z:\python program\WeClone\venv\lib\site-packages\llmtuner\train\tuner.py", line 10, in
from .dpo import run_dpo
File "Z:\python program\WeClone\venv\lib\site-packages\llmtuner\train\dpo_init_.py", line 1, in
from .workflow import run_dpo
File "Z:\python program\WeClone\venv\lib\site-packages\llmtuner\train\dpo\workflow.py", line 9, in
from ...extras.ploting import plot_loss
File "Z:\python program\WeClone\venv\lib\site-packages\llmtuner\extras\ploting.py", line 6, in
from transformers.trainer import TRAINER_STATE_NAME
File "Z:\python program\WeClone\venv\lib\site-packages\transformers\trainer.py", line 59, in
from .data.data_collator import DataCollator, DataCollatorWithPadding, default_data_collator
File "Z:\python program\WeClone\venv\lib\site-packages\transformers\data_init_.py", line 27, in
from .processors import (
File "Z:\python program\WeClone\venv\lib\site-packages\transformers\data\processors_init_.py", line 15, in
from .glue import glue_convert_examples_to_features, glue_output_modes, glue_processors, glue_tasks_num_labels
File "Z:\python program\WeClone\venv\lib\site-packages\transformers\data\processors\glue.py", line 30, in
import tensorflow as tf
File "C:\Python310\lib\site-packages\tensorflow_init_.py", line 38, in
from tensorflow.python.tools import module_util as module_util
File "C:\Python310\lib\site-packages\tensorflow\python_init.py", line 42, in
from tensorflow.python.saved_model import saved_model
File "C:\Python310\lib\site-packages\tensorflow\python\saved_model\saved_model.py", line 20, in
from tensorflow.python.saved_model import builder
File "C:\Python310\lib\site-packages\tensorflow\python\saved_model\builder.py", line 23, in
from tensorflow.python.saved_model.builder_impl import _SavedModelBuilder
File "C:\Python310\lib\site-packages\tensorflow\python\saved_model\builder_impl.py", line 26, in
from tensorflow.python.framework import dtypes
File "C:\Python310\lib\site-packages\tensorflow\python\framework\dtypes.py", line 39, in
_np_float8_e5m2 = pywrap_ml_dtypes.float8_e5m2()
TypeError: Unable to convert function return value to a Python type! The signature was
() -> handle
想问一下大佬用的哪个版本

对话数据不需要整理成QA对吗？

有些奇怪，最近也在看对话聊天记录微调方面，发现很多都是需要整理成QA对，但是实际上Q下一句并不一定是答案，因此很困扰数据该怎么整理