zixiiu / digital_life_server Goto Github PK
View Code? Open in Web Editor NEWYet another voice assistant, but alive.
License: MIT License
Yet another voice assistant, but alive.
License: MIT License
看了一下,需要GPU的模块:
语音识别:可以直接用Windows自带的
情感分析:直接写在prompt里面让chatgpt返回
语音合成:也可以直接用Windows自带的(虽然效果一般,但速度估计会快,而且我试过直接cpu跑vits推理,速度其实也可以接受,毕竟是个400M的小模型)
UE这一部分不是很了解,但这种简单的动画应该不需要很好的GPU也可以完成吧
试了下好像只能连接一台客户端,有办法支持多个设备连接吗,或者说该在哪里修改代码实现多个设备连接一个服务器?
是不是忘记上传requirements.txt了? 先手动下载了
谢谢极客湾!
考虑把模型上传到GoogleDrive吗?百度网盘在境外的下载速度惨不忍睹。。。
支持的 顶一个
客户端在哪个项目里面呀 有没有开源
我的电脑上有以太网、WLAN、WSL,启动时总是会监听 WSL 的 IP,这样手机就没法连接到电脑了。考虑允许手动指定网卡或者直接监听 0.0.0.0
?
大佬,能帮我看看吗? 不知道如何解决
###########
D:\work\Digital_Life_Server>run-gpt3.5-api.bat
Traceback (most recent call last):
File "D:\work\Digital_Life_Server\SocketServer.py", line 16, in
from ASR import ASRService
File "D:\work\Digital_Life_Server\ASR\ASRService.py", line 4, in
from ASR.rapid_paraformer import RapidParaformer
File "D:\work\Digital_Life_Server\ASR\rapid_paraformer_init_.py", line 4, in
from .rapid_paraformer import RapidParaformer
File "D:\work\Digital_Life_Server\ASR\rapid_paraformer\rapid_paraformer.py", line 11, in
from .utils import (CharTokenizer, Hypothesis, ONNXRuntimeError,
File "D:\work\Digital_Life_Server\ASR\rapid_paraformer\utils.py", line 14, in
from typeguard import check_argument_types
ImportError: cannot import name 'check_argument_types' from 'typeguard' (D:\Program Files (x86)\Python\Python310\lib\site-packages\typeguard_init_.py)
做开发太难了。。。。搞半天就被封。有替代的办法吗,例如本地LLM。
运行run-gpt3.5-api.bat后
(venv) E:\BaiduNetdiskDownload\Digital_Life_Server-master>run-gpt3.5-api.bat
2023-05-11 12:29:39,334 INFO Initializing Server...
2023-05-11 12:29:39,336 INFO Initializing ASR Service...
2023-05-11 12:29:49,940 INFO Initializing ChatGPT Service...
2023-05-11 12:29:49,941 INFO chatGPT prompt: paimon35.txt
2023-05-11 12:29:50,136 INFO machine identifier: 00:50:56:C0:00:01
2023-05-11 12:29:50,271 INFO using custom API proxy, with rate limit.
2023-05-11 12:29:50,538 INFO API Chatbot initialized.
2023-05-11 12:29:50,538 INFO Initializing TTS Service for character_paimon...
2023-05-11 12:29:51,624 INFO Loaded checkpoint 'TTS/models/paimon6k_390k.pth' (iteration 8864)
2023-05-11 12:29:51,657 INFO Initializing Sentiment Engine...
2023-05-11 12:29:56,133 INFO Server is listening on 192.168.136.1:38438...
后面该怎么继续
ubuntu或者centos可以部署么
Server整好了又得整Launch,还得整UE。README也是没讲具体操作,小白表示太难了
as so on
How to modify and support with directml
有个小问题,现在UE的GC应该是主推用全局函数IsValid(Object)去检测,成员函数IsValidLowLevel只推荐在引擎内部使用,可以参见这篇:https://unrealcommunity.wiki/memory-management-6rlf3v4i
Could not find a version that satisfies the requirement torch (from versions: none)
有谁把这个项目跑起来了,付费求教,包括如何搭建、如何替换人物,如何替换语音等问题。
付费的,有意私信我
PS G:\Digital_Life_Server\TTS\vits\monotonic_align> python setup.py build_ext --inplace
G:\Digital_Life_Server\TTS\vits\monotonic_align\setup.py:1: DeprecationWarning: The distutils package is deprecated and slated for removal in Python 3.12. Use setuptools or check PEP 632 for potential alternatives
from distutils.core import setup
running build_ext
building 'monotonic_align.core' extension
error: Unable to find vcvarsall.bat
在函数self.chat_gpt.ask_stream(ask_text)
获取到gpt的数据后,如果返回的数据中包含类似๑
颜文字等特殊字符时,传递给TTS发音会导致报错:
GPTService.py[line:96] INFO: ChatGPT Stream Response: 请欣赏!♪♪(๑•̀ㅂ•́)و✧♪♪, @Time 4.34
ERROR: 输入的数据不在转化范围内:๑!
ERROR: Traceback (most recent call last):
File "c:\Users\Digital_Life_Server\test.py", line 130, in listen
self.send_voice(sentence)
File "c:\Users\Digital_Life_Server\test.py", line 177, in send_voice
self.tts.read_save(resp_text, self.tmp_proc_file, self.tts.hps.data.sampling_rate)
File "c:\Users\Digital_Life_Server\TTS\TTService.py", line 55, in read_save
au = self.read(text)
File "c:\Users\Digital_Life_Server\TTS\TTService.py", line 45, in read
stn_tst = get_text(text, self.hps)
File "c:\Users\Digital_Life_Server\TTS\TTService.py", line 24, in get_text
text_norm = text_to_sequence(text, hps.data.text_cleaners)
File "c:\Users\Digital_Life_Server\TTS\vits\text\__init__.py", line 21, in text_to_sequence
clean_text = _clean_text(text, cleaner_names)
File "c:\Users\Digital_Life_Server\TTS\vits\text\__init__.py", line 55, in _clean_text
text = cleaner(text)
File "C:\Users\Digital_Life_Server\TTS/vits\text\cleaners.py", line 34, in chinese_cleaners
text = number_to_chinese(text)
File "C:\Users\Digital_Life_Server\TTS/vits\text\mandarin.py", line 239, in number_to_chinese
text = text.replace(number, cn2an.an2cn(number), 1)
File "C:\Users\Digital_Life_Server\.venv\lib\site-packages\cn2an\an2cn.py", line 40, in an2cn
self.__check_inputs_is_valid(inputs)
File "C:\Users\Digital_Life_Server\.venv\lib\site-packages\cn2an\an2cn.py", line 139, in __check_inputs_is_valid
raise ValueError(f"输入的数据不在转化范围内:{data}!")
ValueError: 输入的数据不在转化范围内:๑!```
self.net_g = SynthesizerTrn(...).cuda()
修改为
self.net_g = SynthesizerTrn(...).cpu()
我替换了视频中打包好的SocketServer.exe为什么就run不起来了。 有什么注意事项嘛?
还是说不光要替换SocketServer.exe
视频中打包好的内容指以下这个
2023-05-31 20:47:40,831 ERROR [WinError 10053] 你的主机中的软件中止了一个已
建立的连接。
2023-05-31 20:47:40,832 ERROR Traceback (most recent call last):
File "D:\Digital_Life_Server\SocketServer.py", line 101, in listen
file = self.__receive_file()
File "D:\Digital_Life_Server\SocketServer.py", line 157, in __receive_file
data = self.conn.recv(1024)
ConnectionAbortedError: [WinError 10053] 你的主机中的软件中止了一个已建立的 连接。
2023-05-31 20:47:40,834 INFO Server is listening on 192.168.206.1:38438...
2023-05-31 20:47:40,835 INFO Connected by ('192.168.206.1', 51163)
切换了vpn后,显示connected了,但是仍然没法打开?
需要用哪个版本的python呀,3.8.16在run-gpt3.5-API.bat步骤会报错
2023-05-19 11:49:30,905 INFO Server is listening on 192.168.8.140:38438...
2023-05-19 11:50:10,612 INFO Initializing Server...
2023-05-19 11:50:10,620 INFO Initializing ASR Service...
2023-05-19 11:50:15,608 INFO Initializing ChatGPT Service...
2023-05-19 11:50:15,610 INFO chatGPT prompt: yunfei35.txt
2023-05-19 11:50:16,162 INFO machine identifier: 7C:50:79:EA:1A:7D
2023-05-19 11:50:16,341 INFO using custom API proxy, with rate limit.
2023-05-19 11:50:16,754 INFO API Chatbot initialized.
2023-05-19 11:50:16,755 INFO Initializing TTS Service for character_yunfei...
2023-05-19 11:50:19,489 INFO Loaded checkpoint 'TTS/models/yunfeimix2_53k.pth' (iteration 1893)
2023-05-19 11:50:19,527 INFO Initializing Sentiment Engine...
2023-05-19 11:50:21,050 INFO Server is listening on 192.168.8.140:38438...
2023-05-19 11:50:48,498 INFO Connected by ('192.168.8.88', 33104)
2023-05-19 11:51:04,357 INFO WAV file received and saved.
2023-05-19 11:51:05,626 INFO ASR Result: ['你好你好收到收到请回答收到请回答']. time used 0.29.
2023-05-19 11:51:06,845 ERROR Expecting value: line 1 column 1 (char 0)
2023-05-19 11:51:06,847 ERROR Traceback (most recent call last):
File "SocketServer.py", line 107, in listen
File "GPT\GPTService.py", line 82, in ask_stream
File "revChatGPT\V3.py", line 215, in ask_stream
File "json_init_.py", line 346, in loads
File "json\decoder.py", line 337, in decode
File "json\decoder.py", line 355, in raw_decode
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
2023-05-19 11:51:06,847 INFO Server is listening on 192.168.8.140:38438...
2023-05-23 12:34:38,359 INFO Initializing Server...
2023-05-23 12:34:38,360 INFO Initializing ASR Service...
2023-05-23 12:34:41,597 INFO Initializing ChatGPT Service...
2023-05-23 12:34:41,598 INFO chatGPT prompt: paimon35.txt
2023-05-23 12:34:41,803 INFO machine identifier: 14:5A:FC:23:CD:47
2023-05-23 12:34:42,041 INFO using custom API proxy, with rate limit.
2023-05-23 12:34:42,315 INFO API Chatbot initialized.
2023-05-23 12:34:42,316 INFO Initializing TTS Service for character_paimon...
2023-05-23 12:34:43,186 INFO Loaded checkpoint 'TTS/models/paimon6k_390k.pth' (iteration 8864)
2023-05-23 12:34:43,218 INFO Initializing Sentiment Engine...
2023-05-23 12:34:43,802 ERROR HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /bert-base-chinese/resolve/main/vocab.txt (Caused by SSLError(SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:997)')))
2023-05-23 12:34:43,809 ERROR Traceback (most recent call last):
File "F:\WorkSpace\python\paimeng\Digital_Life_Server\venv\lib\site-packages\urllib3\connectionpool.py", line 700, in urlopen
self._prepare_proxy(conn)
File "F:\WorkSpace\python\paimeng\Digital_Life_Server\venv\lib\site-packages\urllib3\connectionpool.py", line 996, in _prepare_proxy
conn.connect()
File "F:\WorkSpace\python\paimeng\Digital_Life_Server\venv\lib\site-packages\urllib3\connection.py", line 364, in connect
self.sock = conn = self._connect_tls_proxy(hostname, conn)
File "F:\WorkSpace\python\paimeng\Digital_Life_Server\venv\lib\site-packages\urllib3\connection.py", line 499, in _connect_tls_proxy
socket = ssl_wrap_socket(
File "F:\WorkSpace\python\paimeng\Digital_Life_Server\venv\lib\site-packages\urllib3\util\ssl_.py", line 453, in ssl_wrap_socket
ssl_sock = _ssl_wrap_socket_impl(sock, context, tls_in_tls)
File "F:\WorkSpace\python\paimeng\Digital_Life_Server\venv\lib\site-packages\urllib3\util\ssl_.py", line 495, in _ssl_wrap_socket_impl
return ssl_context.wrap_socket(sock)
File "E:\python\lib\ssl.py", line 512, in wrap_socket
return self.sslsocket_class._create(
File "E:\python\lib\ssl.py", line 1070, in _create
self.do_handshake()
File "E:\python\lib\ssl.py", line 1341, in do_handshake
self._sslobj.do_handshake()
ssl.SSLEOFError: EOF occurred in violation of protocol (_ssl.c:997)
报错如下:
###################################
D:\work\DL_Server>run-gpt3.5-api.bat
2023-05-10 09:42:22,514 INFO Initializing Server...
2023-05-10 09:42:22,514 INFO Initializing ASR Service...
2023-05-10 09:42:28,098 INFO Initializing ChatGPT Service...
2023-05-10 09:42:28,098 INFO chatGPT prompt: paimon35.txt
2023-05-10 09:42:28,519 INFO machine identifier: EE:47:73:19:3B:64
2023-05-10 09:42:28,731 INFO using custom API proxy, with rate limit.
2023-05-10 09:42:29,114 INFO API Chatbot initialized.
2023-05-10 09:42:29,114 INFO Initializing TTS Service for character_paimon...
2023-05-10 09:42:29,940 ERROR
2023-05-10 09:42:29,945 ERROR Traceback (most recent call last):
File "D:\work\DL_Server\SocketServer.py", line 196, in
s = Server(args)
File "D:\work\DL_Server\SocketServer.py", line 86, in init
self.tts = TTService.TTService(*self.char_name[args.character])
File "D:\work\DL_Server\TTS\TTService.py", line 42, in init
_ = utils.load_checkpoint(model, self.net_g, None)
File "D:\work\DL_Server\TTS\vits\utils.py", line 20, in load_checkpoint
assert os.path.isfile(checkpoint_path)
AssertionError
####################################################
语音已经正确在客户端播放,出错是在对话了几轮之后才出的
2023-05-17 19:38:34,106 INFO VITS Synth Done, time used 0.56
2023-05-17 19:38:34,172 INFO Sentiment Engine Infer: 0
2023-05-17 19:38:34,681 INFO WAV SENT, size 2487855
2023-05-17 19:39:14,895 INFO WAV file received and saved.
2023-05-17 19:39:15,100 INFO ASR Result: ['呃我有胡桃叶兰钟梨还有万叶然后还']. time used 0.19.
2023-05-17 19:39:42,389 INFO ChatGPT Response: 好的,根据你提供的信息,我可以给你一些深渊配对建议。首先,你可以选择胡桃
和叶兰作为主DPS,因为他们都有比较高的输出能力和优秀的技能效果。其次,你可以选择钟、梨或者万叶作为辅助DPS或支援角色,他们都有自己的特点和优势,可以为队伍提供额外的输出或支援能力。最后,你需要根据深渊的要求和敌人的属性来选择合适的角色,比如选择具有火属性或雷属性的角色来对付水属性或冰属性的敌人。总的来说,你的队伍已经比较平衡了,只需要根据具体情况做出一些微调就可以了。, time used 27.29
2023-05-17 19:39:43,136 INFO VITS Synth Done, time used 0.74
2023-05-17 19:39:43,220 INFO Sentiment Engine Infer: 0
2023-05-17 19:39:43,733 INFO WAV SENT, size 3402799
2023-05-17 19:39:44,238 INFO WAV file received and saved.
H:\AITakagi_Digital_Life\Digital_Life_Server\SocketServer.py:184: UserWarning: PySoundFile failed. Trying audioread instead.
y, sr = librosa.load(self.tmp_recv_file, sr=None, mono=False)
H:\AITakagi_Digital_Life\Digital_Life_Server\venv\lib\site-packages\librosa\core\audio.py:184:FutureWarning:librosa.core.audio.__audioread_load
Deprecated as of librosa version 0.10.0.
It will be removed in librosa version 1.0.
y, sr_native = __audioread_load(path, offset, duration, dtype)
2023-05-17 19:39:44,674 ERROR
2023-05-17 19:39:44,705 ERROR Traceback (most recent call last):
File "H:\AITakagi_Digital_Life\Digital_Life_Server\venv\lib\site-packages\librosa\core\audio.py", line 176, in load
y, sr_native = __soundfile_load(path, offset, duration, dtype)
File "H:\AITakagi_Digital_Life\Digital_Life_Server\venv\lib\site-packages\librosa\core\audio.py", line 209, in __soundfile_load
context = sf.SoundFile(path)
File "H:\AITakagi_Digital_Life\Digital_Life_Server\venv\lib\site-packages\soundfile.py", line 658, in init
self._file = self._open(file, mode_int, closefd)
File "H:\AITakagi_Digital_Life\Digital_Life_Server\venv\lib\site-packages\soundfile.py", line 1216, in _open
raise LibsndfileError(err, prefix="Error opening {0!r}: ".format(self.name))
soundfile.LibsndfileError: Error opening 'tmp/server_received.wav': Format not recognised.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "H:\AITakagi_Digital_Life\Digital_Life_Server\SocketServer.py", line 106, in listen
ask_text = self.process_voice()
File "H:\AITakagi_Digital_Life\Digital_Life_Server\SocketServer.py", line 184, in process_voice
y, sr = librosa.load(self.tmp_recv_file, sr=None, mono=False)
File "H:\AITakagi_Digital_Life\Digital_Life_Server\venv\lib\site-packages\librosa\core\audio.py", line 184, in load
y, sr_native = __audioread_load(path, offset, duration, dtype)
File "H:\AITakagi_Digital_Life\Digital_Life_Server\venv\lib\site-packages\decorator.py", line 232, in fun return caller(func, *(extras + args), **kw)
File "H:\AITakagi_Digital_Life\Digital_Life_Server\venv\lib\site-packages\librosa\util\decorators.py", line 60, in __wrapper
return func(*args, **kwargs)
File "H:\AITakagi_Digital_Life\Digital_Life_Server\venv\lib\site-packages\librosa\core\audio.py", line 241, in _audioread_load
reader = audioread.audio_open(path)
File "H:\AITakagi_Digital_Life\Digital_Life_Server\venv\lib\site-packages\audioread_init.py", line 132, in audio_open
raise NoBackendError()
audioread.exceptions.NoBackendError
(base) D:\Digital_Life_Server>run-gpt3.5-api.bat
D:\Digital_Life_Server\venv\lib\site-packages\revChatGPT_init_.py:27: UserWarning: The current Python is not a recommended version, 3.10+ is recommended
import("warnings").warn(
Traceback (most recent call last):
File "D:\Digital_Life_Server\SocketServer.py", line 18, in
from TTS import TTService
File "D:\Digital_Life_Server\TTS\TTService.py", line 9, in
import torch
ModuleNotFoundError: No module named 'torch'
(base) D:\Digital_Life_Server>python
Python 3.9.13 (main, Aug 25 2022, 23:51:50) [MSC v.1916 64 bit (AMD64)] :: Anaconda, Inc. on win32
Type "help", "copyright", "credits" or "license" for more information.
import torch
ImportError: cannot import name '_tiktoken' from partially initialized module 'tiktoken' (most likely due to a circular import) (D:\Anaconda3\envs\digital_life\lib\site-packages\tiktoken\__init__.py)
如上,我在导入tiktoken时发生了循环导入的错误,有人遇到这个问题吗?
2023-05-08 20:53:08,741 INFO Initializing Server...
2023-05-08 20:53:08,744 INFO Initializing ASR Service...
2023-05-08 20:53:11,130 INFO Initializing ChatGPT Service...
2023-05-08 20:53:11,131 INFO chatGPT prompt: paimon35.txt
2023-05-08 20:53:11,201 INFO machine identifier: 50:EB:F6:ED:94:FE
2023-05-08 20:53:11,299 INFO using custom API proxy, with rate limit.
2023-05-08 20:53:11,485 INFO API Chatbot initialized.
2023-05-08 20:53:11,485 INFO Initializing TTS Service for character_paimon...
2023-05-08 20:53:12,166 ERROR Error(s) in loading state_dict for SynthesizerTrn:
size mismatch for enc_p.emb.weight: copying a param with shape torch.Size([52, 192]) from checkpoint, the shape in current model is torch.Size([178, 192]).
2023-05-08 20:53:12,168 ERROR Traceback (most recent call last):
File "h:\Digital_Life_Server-master\SocketServer.py", line 196, in <module>
s = Server(args)
File "h:\Digital_Life_Server-master\SocketServer.py", line 86, in __init__
self.tts = TTService.TTService(*self.char_name[args.character])
File "h:\Digital_Life_Server-master\TTS\TTService.py", line 42, in __init__
_ = utils.load_checkpoint(model, self.net_g, None)
File "h:\Digital_Life_Server-master\TTS\vits\utils.py", line 40, in load_checkpoint
model.load_state_dict(new_state_dict)
File "C:\Users\colwzq\anaconda3\envs\digital-life\lib\site-packages\torch\nn\modules\module.py", line 2041, in load_state_dict
raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for SynthesizerTrn:
size mismatch for enc_p.emb.weight: copying a param with shape torch.Size([52, 192]) from checkpoint, the shape in current model is torch.Size([178, 192]).
yunfei和paimon都试过了,报错一致
2023-05-26 18:03:04,765 INFO Initializing Server...
2023-05-26 18:03:04,765 INFO Initializing ASR Service...
服务器进程已退出!
能不能提供下这些文件,谢谢分享!
cmd.exe /c run-gpt3.5-api.bat
Traceback (most recent call last): File "E:\Digital Human\Digital_Life_Server\SocketServer.py", line 18, in from TTS import TTService File "E:\Digital Human\Digital_Life_Server\TTS\TTService.py", line 14, in from TTS.vits.models import SynthesizerTrn File "E:\Digital Human\Digital_Life_Server\TTS\vits\models.py", line 10, in import monotonic_align File "E:\Digital Human\Digital_Life_Server\TTS/vits\monotonic_align_init_.py", line 3, in from .monotonic_align.core import maximum_path_c
ModuleNotFoundError: No module named 'monotonic_align.monotonic_align.core'
已经把core.cp38-win_amd64.pyd文件放在\TTS\vits\monotonic_align\monotonic_align文件夹下了,依然报错
严重性 代码 说明 项目 文件 行 禁止显示状态
错误 BC30456 “Form1”不是“Launcher”的成员。 Launcher C:\Users\lzc\Documents\DL_Launcher-master\DL_Launcher-master\Launcher\My Project\Application.Designer.vb 35 活动
这个form1应该改成什么
需要加自动转格式的code?
用了MoeGoe
后显示报错了
这应该使用哪种的vits环境来推理啊
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.