zhaoyingjun/chatbot

ChatGPT带火了聊天机器人，主流的趋势都调整到了GPT类模式，本项目也与时俱进，会在近期更新GPT类版本。基于本项目和自己的语料可以训练出自己想要的聊天机器人，用于智能客服、在线问答、闲聊等场景。

chatbot

ChatGPT带火了聊天机器人，主流的趋势都调整到了GPT类模式，本项目也与时俱进，会在近期更新更新GPT类版本。 这是一个可以使用自己语料进行训练的中文聊天机器人项目，欢迎大家实践交流以及Star、Fork。

Seq2Seq版本效果参考（训练进度50%）

RoadMap:

V1.1:Update:2024-09-30

1)增加MindSpore版本，优先在MindSpore版本上引入GPT模型，RLHF等特性。

2）整体工程架构分为Seq2Seq和GPT两大分支，继续保持多AI框架版本演进。

V1.2:Update:2024-12-30(Maybe)

1）实现类似mini-GPT4的功能，可以进行图文多模态的对话，主要提升趣味性和丰富性。

2）增强分布式集群训练相关能力和RLHF等特性。

seq2seq版本代码执行顺序

大家可以使用小黄鸡的语料，下载地址https://github.com/zhaoyingjun/chatbot/blob/master/chineseChatbotWeb-tf2.0/seq2seqChatbot/train_data/xiaohuangji50w_nofenci.conv

1）、在下载好代码和语料之后，将语料文件放入train_data目录下，超参配置在config/seq2seq.ini文件中配置。。

2）、按照数据预处理器（data_utls.py)-->execute.py(执行器)-->app.py（可视化对话模块）的顺序执行就可以了。

3）、大规模分布式训练版本，参照horovod的启动方式 horovodrun -np n -H host1_ip:port,host2_ip:port,hostn_ip:port python3 excute.py

建议训练环境配置

ubuntu==18.04
python==3.6

TF2.X:

tensorflow==2.6.0

flask==0.11.1

horovod==0.24(分布式训练)

pytorch:

torch==1.11.0

flask==0.11.1

开源交流、联系方式

QQ：934389697

chatbot's Issues

heartbeat issue

hi，我跑你代码execute.py时，跑完后显示：
####################
Mode : train

Preparing data in working_dir
working_dir/train.enc 词汇表大小: 20000
working_dir/train.dec 词汇表大小: 20000
对话转向量...
对话转向量...
对话转向量...
对话转向量...
Creating 3 layers of 256 units.
Created model with fresh parameters.
Killed
############################
然后我执行根目录下的app.py, 结果一直显示heartbeat，每分钟显示一天，都显示了5个小时了
2018-03-06 22:36:23 - heartbeat
2018-03-06 22:37:23 - heartbeat
2018-03-06 22:38:23 - heartbeat
2018-03-06 22:39:23 - heartbeat
2018-03-06 22:40:23 - heartbeat

请问这是正常的吗？我是继续等待还是需要修改什么地方？

資料連結失效

點進去說過期或是頁面不在了

训练完模型，该如何在服务器上测试

@zhaoyingjun , hi ,您好，我也训练完后，在服务器测试，结果如下，

• Serving Flask app "app" (lazy loading)
• Environment: production
• WARNING: Do not use the development server in a production environment.
• Use a production WSGI server instead.
• Debug mode: off
• Running on http://12.12.12.12:1080/ (Press CTRL+C to quit)

然后在浏览器打不开http://12.12.12.12:1080/，请问这个怎么解决的

• 上面的12.12.12.12地址代替了我自己的服务器地址

将字典规模调高后，enc_vocab_size=dec_vocab_size=50000, error!

请问这是哪里出了问题？

Traceback (most recent call last):
File "app.py", line 67, in
sess, model, enc_vocab, rev_dec_vocab = execute.init_session(sess, conf='seq
2seq_serve.ini')
File "C:\Users\Administrator\Downloads\chatbot-master\chatbot-master\execute.p
y", line 211, in init_session
gConfig = get_config(conf)
File "C:\Users\Administrator\Downloads\chatbot-master\chatbot-master\execute.p
y", line 42, in get_config
parser.read(config_file)
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\config
parser.py", line 697, in read
self._read(fp, filename)
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36\lib\config
parser.py", line 1015, in _read
for lineno, line in enumerate(fp, start=1):
UnicodeDecodeError: 'gbk' codec can't decode byte 0xae in position 69: illegal m
ultibyte sequence

自定义的语料应该是什么样的呢？问答型的可以吗

Internal Server Error

运行app.py时出现这个错误，请问怎么回事？The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application.
127.0.0.1 - - [07/Dec/2018 08:51:39] "GET / HTTP/1.1" 500 -
127.0.0.1 - - [07/Dec/2018 08:51:39] "GET /favicon.ico HTTP/1.1" 404 -
2018-12-07 08:52:09 - heartbeat

訓練出來的ckpt.data及ckpt.meta大小都固定是正常嗎?

您好，我執行execute.py之後，不管我的train.enc跟train.dec有多少對話筆數
訓練出來的seq2seq.ckpt容量都一樣，這是正常的嗎~還是有設定只取前幾筆對話，導致訓練出來的模型大小都一樣?

seqGAN版本,disc的数据怎么得到的呢，disc_data/train.query，disc_data/train.gen等

seq2seq推理部分有点问题

execute.py下的decode_line（）函数会把中文词转为二进制表示，所以在词字典中是找不到的，token_ids一直是[3,3,3,3…]

需要訓練多久?perplexity 多少算正常?

一開始
global step 2100 learning rate 0.5000 step-time 0.52 perplexity 151.84
現在
global step 18000 learning rate 0.4950 step-time 0.65 perplexity 1.30
看起來下降滿多的，但是容量都固定就覺得有點怪~
想請教perplexity 要多少才算合理?

另外想請教 @zhaoyingjun 一下，要訓練多久或是step到達多少才會訓練結束?
感謝~

我想用这个开发一个智能营销的系统

现在遇到的问题是
（1）会话管理 ，主流程 和多个副流程 主流程是主要介绍和受理业务的 副流程是 回答用户的各种各样的问题的。。。
这个能实现吗

训练时轮训怎么退出 看代码while True 没有退出语句

训练时轮训怎么退出 看代码while True 没有退出语句

已经训练完成，但是要怎么在本地运行起来呢

我想看看训练效果，直接执行./webui/app.py会打不开0.0.0.0:8080。因为没有使用服务器的经验，希望您能告知一下。

执行 execute.py 时报TypeError: can't pickle _thread.RLock objects

请问下，每次在create_model下到model = seq2seq_model.Seq2SeqModel( gConfig['enc_vocab_size'], gConfig['dec_vocab_size'], _buckets, gConfig['layer_size'], gConfig['num_layers'], gConfig['max_gradient_norm'], gConfig['batch_size'], gConfig['learning_rate'], gConfig['learning_rate_decay_factor'], forward_only=forward_only)这步时，就会TypeError: can't pickle _thread.RLock objects，请问有遇到过吗？如何解决的呢？

train.enc要从哪里搞？

提示找不到文件

seqGan资源缺失了

目录中并无以下资源：
train_enc = "./gen_data/train.query"
train_dec = "./gen_data/train.answer"
test_enc = "./gen_data/test.query"
test_dec = "./gen_data/test.answer"
resource_data = "./data_source/conv.txt"

回答语句出现乱码

您好，我在ubuntu18.04虚拟机下通过了训练，运行，访问服务器后出现对应界面，发送语句后得到了这样的乱码 ç � ® å ¾ � å ¹ ² ä º �
请问应该如何解决？

想问下编码问题

rev_dec_vocab[output] 返回给我的是 å � « ç ´ § å ¼ 我怎么转成中文

seqGan的版本呢？求更新啊

seqGAN，execute.py出现list index out of range的问题,

您好，我训练seqGAN中的模型，出现下面的结果和错误，请问这应该如何解决，谢谢

• ==================Update Discriminator: 199=====================
• ==============================mc_search:False===================================
• ==================Update Generator: 199=========================
• =============================mc_search:True====================================

这里的Discriminator出现的结果总是False，同时，报以下的错误

• File "execute.py", line 198, in al_train

• print(" ".join([tf.compat.as_str(rev_vocab[output]) for output in train_answer[i]]))
  

• File "execute.py", line 198, in

• print(" ".join([tf.compat.as_str(rev_vocab[output]) for output in train_answer[i]]))
  

• IndexError: list index out of range

请问，这是哪里出现了问题，谢谢

您好

File "D:\Anaconda3\envs\tensorflow-gpu\lib\copy.py", line 169, in deepcopy
rv = reductor(4)
TypeError: can't pickle _thread.RLock objects
这个怎么解决呢，谢谢

增大词典大小后，启动app.py报错？

启动app.py报错：

tensorflow.python.framework.errors_impl.InvalidArgumentError: Assign requires shapes of both tensors to match. lhs shape= [256,20000] rhs shape= [256,100000]
	 [[Node: save/Assign_21 = Assign[T=DT_FLOAT, _class=["loc:@proj_w"], use_locking=true, validate_shape=true, _device="/job:localhost/replica:0/task:0/device:CPU:0"](proj_w, save/RestoreV2:21)]]

请问这是什么原因？

关于导入配置文件时gbk编码出错问题的解决

报错详细信息如下
Traceback (most recent call last):
File "data_utls.py", line 13, in
gConfig=getConfig.get_config()
File "C:\Users\53194\Desktop\chatbot-master\seq2seqChatbot\getConfig.py", line 29, in get_config
parser.read(config_file)
File "C:\Users\53194\AppData\Local\Programs\Python\Python37-32\lib\configparser.py", line 696, in read
self._read(fp, filename)
File "C:\Users\53194\AppData\Local\Programs\Python\Python37-32\lib\configparser.py", line 1014, in _read
for lineno, line in enumerate(fp, start=1):
UnicodeDecodeError: 'gbk' codec can't decode byte 0xae in position 69: illegal multibyte sequence

明显是在读seq2seq.ini时出错，经测试发现是中文注释问题。如果发现该错误，删去该文件中所有中文注释即可。

应用场景问题

您好：
想问一下seq2seq用来做问答系统，与KBQA相比效果如何呢？目前在做问答系统的选型工作，KBQA对问答库的依赖比较大，等于是基于分词去进行检索，个人并不看好这种方式。所以还请您指点一下，谢谢！

为什么训练出来的机器人只回答简单的表情符号或者=号？

基于之前的电影语料，训练600w次，也是只回答固定的毫无逻辑的话或者表情符号
换了黄鸡语料也只回答简单的表情符号或者=号
求问需要参数特殊设置吗？
以及黄鸡训练成如图的效果花了多久时间？
（我是基于windows开发的，将遇到的gbk问题都改为了utf-8）

欢迎交流

试验了下，web界面真心不错，效果真烂

训练效果的问题

enc_vocab_size = 100000
dec_vocab_size = 100000
max_train_data_size = 200000

我在训练50w步后大概接近 1.1这样，但发现回答效果还是重复，中间有续训，请问是还要提高词向量大小吗，还是我中间出了什么问题

[seqGanChatbot] 調用 gen_pre_train()時，報錯

一開始，是於utils/data_util.py中，vocab_file.write(w + "\n")這一行報錯，顯示bytes不能與字串"\n"連結在一起存檔，所以修改為vocab_file.write(str(w) + "\n")順利儲存gen_data/train.answer、train.query，test.query，以及test.answer四個檔案，接著，執行execute.py 在 # step_1 training gen model，調用 gen_pre_train()時，報錯(有產生在gen_data路徑下vocab2500.all，有64.6KB)，報錯所以終止執行，錯誤訊息如下：File "/seqGanChatbot/utils/data_utils.py", line 52, in basic_tokenizer，查找後，發現words.extend(re.split(_WORD_SPLIT, space_separated_fragment))這行，會丟出: cannot use a bytes pattern on a string-like object例外，再看gen_data/vocab2500.all裡，確實都是存成了 b'\xe5\x9c\x9f'這樣的bytes換行符號，能否請您協助查看呢? 感謝!!

rev_vocab always out of range?

File "execute.py", line 338, in main
gen_disc()
File "execute.py", line 53, in gen_disc
gens.decoder(gen_config)
File "/home/bigdata/Leo/seqGanChatbot/gen/generator.py", line 266, in decoder
answer_str = " ".join([str(rev_vocab[an]) for an in answer[:-1]])
File "/home/bigdata/Leo/seqGanChatbot/gen/generator.py", line 266, in
answer_str = " ".join([str(rev_vocab[an]) for an in answer[:-1]])
IndexError: list index out of range
any clue to fix it? thx

got encoding problem when running data_util.py

After downloaded the project and put the conversation data file to the right place without any modification, i ran data_util.py but got errors message: 'ascii' codec can't decode byte 0xe7 in position 4: ordinal not in range(128).
I checked data_util.py and saw "# coding=utf-8" on the head. It seemed didn't work.
I added the code below:
import sys
if("utf-8"!=sys.getdefaultencoding()):
reload(sys)
sys.setdefaultencoding("utf-8")
but the error continued

训练结果出现大量unk

你好，我在使用默认的训练集进行训练，大概进行了14000steps后（preprlexity<10）后，对话质量好像很差，而且出现大量unk符号，按照建议我已经将enc_vocab_size和dec_vocab_size加大到100000，但好像没有改善。 我不知道问题处在哪。

用的词嵌入吗

用的词嵌入吗还是one-hot啊？求GAN的更新啊，哈哈

请问下，可以手动停止训练吗？

我在执行python execute.py训练过程中，感觉preplexity已经不往下降了，可以手动停止训练么？还是说如果preplexity收敛不到1就不停止？

how to stop train

how to stop train? 在train函数中，where True 那怎么结束这个训练尼？我没看到那个地方可以退出循环的呀。不懂，请帮解释下。

训练效果不佳，求解

按照建议将词典大小增加到100000，然后用自带语料库训练后出现大量UNK，后来自己找了小黄鸡语料，perplexity降到了3左右就基本不下降了，但是回答的效果很差，没有任何逻辑，我想问下是为什么...语料的问题吗？

请问下，模型有带attention机制吗？

请问回复都是一些英语类似的字符，这是什么原因？

按这个（data_utls.py)-->execute.py(执行器)-->app.py（可视化对话模块）顺序执行的，然后结果：

训练时报错

你好，我按照步骤做到训练这一步，报错如下：
Traceback (most recent call last):
File "E:/work/hc-chatbot/开源代码/chatbot-master1/execute.py", line 260, in
train()
File "E:/work/hc-chatbot/开源代码/chatbot-master1/execute.py", line 125, in train
model = create_model(sess, False)
File "E:/work/hc-chatbot/开源代码/chatbot-master1/execute.py", line 95, in create_model
model = seq2seq_model.Seq2SeqModel( gConfig['enc_vocab_size'], gConfig['dec_vocab_size'], _buckets, gConfig['layer_size'], gConfig['num_layers'], gConfig['max_gradient_norm'], gConfig['batch_size'], gConfig['learning_rate'], gConfig['learning_rate_decay_factor'], forward_only=forward_only)
File "E:\work\hc-chatbot\开源代码\chatbot-master1\seq2seq_model.py", line 149, in init
softmax_loss_function=softmax_loss_function)
File "E:\Users\hczpp\Anaconda3\lib\site-packages\tensorflow\contrib\legacy_seq2seq\python\ops\seq2seq.py", line 1209, in model_with_buckets
decoder_inputs[:bucket[1]])
File "E:\work\hc-chatbot\开源代码\chatbot-master1\seq2seq_model.py", line 148, in
lambda x, y: seq2seq_f(x, y, False),
File "E:\work\hc-chatbot\开源代码\chatbot-master1\seq2seq_model.py", line 112, in seq2seq_f
feed_previous=do_decode)
File "E:\Users\hczpp\Anaconda3\lib\site-packages\tensorflow\contrib\legacy_seq2seq\python\ops\seq2seq.py", line 851, in embedding_attention_seq2seq
encoder_cell = copy.deepcopy(cell)
File "E:\Users\hczpp\Anaconda3\lib\copy.py", line 166, in deepcopy
y = copier(memo)
File "E:\Users\hczpp\Anaconda3\lib\site-packages\tensorflow\python\layers\base.py", line 809, in deepcopy
setattr(result, k, copy.deepcopy(v, memo))
File "E:\Users\hczpp\Anaconda3\lib\copy.py", line 155, in deepcopy
y = copier(x, memo)
File "E:\Users\hczpp\Anaconda3\lib\copy.py", line 243, in _deepcopy_dict
y[deepcopy(key, memo)] = deepcopy(value, memo)
File "E:\Users\hczpp\Anaconda3\lib\copy.py", line 166, in deepcopy
y = copier(memo)
File "E:\Users\hczpp\Anaconda3\lib\site-packages\tensorflow\python\layers\base.py", line 809, in deepcopy
setattr(result, k, copy.deepcopy(v, memo))
File "E:\Users\hczpp\Anaconda3\lib\copy.py", line 155, in deepcopy
y = copier(x, memo)
File "E:\Users\hczpp\Anaconda3\lib\copy.py", line 243, in _deepcopy_dict
y[deepcopy(key, memo)] = deepcopy(value, memo)
File "E:\Users\hczpp\Anaconda3\lib\copy.py", line 182, in deepcopy
y = _reconstruct(x, rv, 1, memo)
File "E:\Users\hczpp\Anaconda3\lib\copy.py", line 297, in _reconstruct
state = deepcopy(state, memo)
File "E:\Users\hczpp\Anaconda3\lib\copy.py", line 155, in deepcopy
y = copier(x, memo)
File "E:\Users\hczpp\Anaconda3\lib\copy.py", line 243, in _deepcopy_dict
y[deepcopy(key, memo)] = deepcopy(value, memo)
File "E:\Users\hczpp\Anaconda3\lib\copy.py", line 182, in deepcopy
y = _reconstruct(x, rv, 1, memo)
File "E:\Users\hczpp\Anaconda3\lib\copy.py", line 297, in _reconstruct
state = deepcopy(state, memo)
File "E:\Users\hczpp\Anaconda3\lib\copy.py", line 155, in deepcopy
y = copier(x, memo)
File "E:\Users\hczpp\Anaconda3\lib\copy.py", line 243, in _deepcopy_dict
y[deepcopy(key, memo)] = deepcopy(value, memo)
File "E:\Users\hczpp\Anaconda3\lib\copy.py", line 155, in deepcopy
y = copier(x, memo)
File "E:\Users\hczpp\Anaconda3\lib\copy.py", line 218, in _deepcopy_list
y.append(deepcopy(a, memo))
File "E:\Users\hczpp\Anaconda3\lib\copy.py", line 182, in deepcopy
y = _reconstruct(x, rv, 1, memo)
File "E:\Users\hczpp\Anaconda3\lib\copy.py", line 297, in _reconstruct
state = deepcopy(state, memo)
File "E:\Users\hczpp\Anaconda3\lib\copy.py", line 155, in deepcopy
y = copier(x, memo)
File "E:\Users\hczpp\Anaconda3\lib\copy.py", line 243, in _deepcopy_dict
y[deepcopy(key, memo)] = deepcopy(value, memo)
File "E:\Users\hczpp\Anaconda3\lib\copy.py", line 155, in deepcopy
y = copier(x, memo)
File "E:\Users\hczpp\Anaconda3\lib\copy.py", line 218, in _deepcopy_list
y.append(deepcopy(a, memo))
File "E:\Users\hczpp\Anaconda3\lib\copy.py", line 182, in deepcopy
y = _reconstruct(x, rv, 1, memo)
File "E:\Users\hczpp\Anaconda3\lib\copy.py", line 297, in _reconstruct
state = deepcopy(state, memo)
File "E:\Users\hczpp\Anaconda3\lib\copy.py", line 155, in deepcopy
y = copier(x, memo)
File "E:\Users\hczpp\Anaconda3\lib\copy.py", line 243, in _deepcopy_dict
y[deepcopy(key, memo)] = deepcopy(value, memo)
File "E:\Users\hczpp\Anaconda3\lib\copy.py", line 182, in deepcopy
y = _reconstruct(x, rv, 1, memo)
File "E:\Users\hczpp\Anaconda3\lib\copy.py", line 297, in _reconstruct
state = deepcopy(state, memo)
File "E:\Users\hczpp\Anaconda3\lib\copy.py", line 155, in deepcopy
y = copier(x, memo)
File "E:\Users\hczpp\Anaconda3\lib\copy.py", line 243, in _deepcopy_dict
y[deepcopy(key, memo)] = deepcopy(value, memo)
File "E:\Users\hczpp\Anaconda3\lib\copy.py", line 182, in deepcopy
y = _reconstruct(x, rv, 1, memo)
File "E:\Users\hczpp\Anaconda3\lib\copy.py", line 291, in _reconstruct
args = deepcopy(args, memo)
File "E:\Users\hczpp\Anaconda3\lib\copy.py", line 155, in deepcopy
y = copier(x, memo)
File "E:\Users\hczpp\Anaconda3\lib\copy.py", line 223, in _deepcopy_tuple
y = [deepcopy(a, memo) for a in x]
File "E:\Users\hczpp\Anaconda3\lib\copy.py", line 223, in
y = [deepcopy(a, memo) for a in x]
File "E:\Users\hczpp\Anaconda3\lib\copy.py", line 155, in deepcopy
y = copier(x, memo)
File "E:\Users\hczpp\Anaconda3\lib\copy.py", line 218, in _deepcopy_list
y.append(deepcopy(a, memo))
File "E:\Users\hczpp\Anaconda3\lib\copy.py", line 182, in deepcopy
y = _reconstruct(x, rv, 1, memo)
File "E:\Users\hczpp\Anaconda3\lib\copy.py", line 297, in _reconstruct
state = deepcopy(state, memo)
File "E:\Users\hczpp\Anaconda3\lib\copy.py", line 155, in deepcopy
y = copier(x, memo)
File "E:\Users\hczpp\Anaconda3\lib\copy.py", line 243, in _deepcopy_dict
y[deepcopy(key, memo)] = deepcopy(value, memo)
File "E:\Users\hczpp\Anaconda3\lib\copy.py", line 155, in deepcopy
y = copier(x, memo)
File "E:\Users\hczpp\Anaconda3\lib\copy.py", line 218, in _deepcopy_list
y.append(deepcopy(a, memo))
File "E:\Users\hczpp\Anaconda3\lib\copy.py", line 182, in deepcopy
y = _reconstruct(x, rv, 1, memo)
File "E:\Users\hczpp\Anaconda3\lib\copy.py", line 297, in _reconstruct
state = deepcopy(state, memo)
File "E:\Users\hczpp\Anaconda3\lib\copy.py", line 155, in deepcopy
y = copier(x, memo)
File "E:\Users\hczpp\Anaconda3\lib\copy.py", line 243, in _deepcopy_dict
y[deepcopy(key, memo)] = deepcopy(value, memo)
File "E:\Users\hczpp\Anaconda3\lib\copy.py", line 155, in deepcopy
y = copier(x, memo)
File "E:\Users\hczpp\Anaconda3\lib\copy.py", line 218, in _deepcopy_list
y.append(deepcopy(a, memo))
File "E:\Users\hczpp\Anaconda3\lib\copy.py", line 182, in deepcopy
y = _reconstruct(x, rv, 1, memo)
File "E:\Users\hczpp\Anaconda3\lib\copy.py", line 297, in _reconstruct
state = deepcopy(state, memo)
File "E:\Users\hczpp\Anaconda3\lib\copy.py", line 155, in deepcopy
y = copier(x, memo)
File "E:\Users\hczpp\Anaconda3\lib\copy.py", line 243, in _deepcopy_dict
y[deepcopy(key, memo)] = deepcopy(value, memo)
File "E:\Users\hczpp\Anaconda3\lib\copy.py", line 182, in deepcopy
y = _reconstruct(x, rv, 1, memo)
File "E:\Users\hczpp\Anaconda3\lib\copy.py", line 297, in _reconstruct
state = deepcopy(state, memo)
File "E:\Users\hczpp\Anaconda3\lib\copy.py", line 155, in deepcopy
y = copier(x, memo)
File "E:\Users\hczpp\Anaconda3\lib\copy.py", line 243, in _deepcopy_dict
y[deepcopy(key, memo)] = deepcopy(value, memo)
File "E:\Users\hczpp\Anaconda3\lib\copy.py", line 155, in deepcopy
y = copier(x, memo)
File "E:\Users\hczpp\Anaconda3\lib\copy.py", line 218, in _deepcopy_list
y.append(deepcopy(a, memo))
File "E:\Users\hczpp\Anaconda3\lib\copy.py", line 155, in deepcopy
y = copier(x, memo)
File "E:\Users\hczpp\Anaconda3\lib\copy.py", line 223, in _deepcopy_tuple
y = [deepcopy(a, memo) for a in x]
File "E:\Users\hczpp\Anaconda3\lib\copy.py", line 223, in
y = [deepcopy(a, memo) for a in x]
File "E:\Users\hczpp\Anaconda3\lib\copy.py", line 155, in deepcopy
y = copier(x, memo)
File "E:\Users\hczpp\Anaconda3\lib\copy.py", line 243, in _deepcopy_dict
y[deepcopy(key, memo)] = deepcopy(value, memo)
File "E:\Users\hczpp\Anaconda3\lib\copy.py", line 182, in deepcopy
y = _reconstruct(x, rv, 1, memo)
File "E:\Users\hczpp\Anaconda3\lib\copy.py", line 297, in _reconstruct
state = deepcopy(state, memo)
File "E:\Users\hczpp\Anaconda3\lib\copy.py", line 155, in deepcopy
y = copier(x, memo)
File "E:\Users\hczpp\Anaconda3\lib\copy.py", line 243, in _deepcopy_dict
y[deepcopy(key, memo)] = deepcopy(value, memo)
File "E:\Users\hczpp\Anaconda3\lib\copy.py", line 182, in deepcopy
y = _reconstruct(x, rv, 1, memo)
File "E:\Users\hczpp\Anaconda3\lib\copy.py", line 306, in _reconstruct
y.dict.update(state)
AttributeError: 'NoneType' object has no attribute 'update'

centos下解析'data/dgk_shooter_z.conv'全是乱码

centos下解析'data/dgk_shooter_z.conv'全是乱码

tensorflow.python.framework.errors_impl.NotFoundError

tensorflow.python.framework.errors_impl.NotFoundError: NewRandomAccessFile failed to Create/Open: ./gen_data/train.answer : ϵͳ\udcd5Ҳ\udcbb\udcb5\udcbdָ\udcb6\udca8\udcb5\udcc4·\udcbe\udcb6\udca1\udca3`

执行 seqGanChatbot\execute.py 时报此错误

perplexity接近1，但回答都重複

不好意思，想再請問一下
我訓練大概step到十萬左右，perplexity都很接近1了，大概1.0X
但執行app.py之後的回覆句子都會是同一兩句(非預設的''^_^'')
請問這個原因可能是什麼?

訓練出來的ckpt.data及ckpt.meta大小都固定是正常嗎?

目前我這個從step0到現在，data都固定71365KB，meta都39960KB

另一個之前訓練的已經step八萬多，容量也都一樣

但可以看出兩個的train.dec並不一樣大小，而且正常data跟meta應該會越來越多才對吧~~
會是因為seq2seq_serve.ini有個max_train_data_size值設定50000的關係嗎?

TypeError: sampled_loss() got an unexpected keyword argument 'logits'

我的tensorflow版本是1.6，发生如下问题，请大牛解答,谢谢

2018-05-28 11:16:27.772360: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Creating 3 layers of 256 units.
Traceback (most recent call last):
File "/Users/jiangwei/PycharmProjects/chatbot/chatbot_master/execute.py", line 259, in
train()
File "/Users/jiangwei/PycharmProjects/chatbot/chatbot_master/execute.py", line 124, in train
model = create_model(sess, False)
File "/Users/jiangwei/PycharmProjects/chatbot/chatbot_master/execute.py", line 94, in create_model
model = seq2seq_model.Seq2SeqModel( gConfig['enc_vocab_size'], gConfig['dec_vocab_size'], _buckets, gConfig['layer_size'], gConfig['num_layers'], gConfig['max_gradient_norm'], gConfig['batch_size'], gConfig['learning_rate'], gConfig['learning_rate_decay_factor'], forward_only=forward_only)
File "/Users/jiangwei/PycharmProjects/chatbot/chatbot_master/seq2seq_model.py", line 149, in init
softmax_loss_function=softmax_loss_function)
File "/Users/jiangwei/anaconda/envs/tensorflow-1.6-py3.5/lib/python3.5/site-packages/tensorflow/contrib/legacy_seq2seq/python/ops/seq2seq.py", line 1224, in model_with_buckets
softmax_loss_function=softmax_loss_function))
File "/Users/jiangwei/anaconda/envs/tensorflow-1.6-py3.5/lib/python3.5/site-packages/tensorflow/contrib/legacy_seq2seq/python/ops/seq2seq.py", line 1137, in sequence_loss
softmax_loss_function=softmax_loss_function))
File "/Users/jiangwei/anaconda/envs/tensorflow-1.6-py3.5/lib/python3.5/site-packages/tensorflow/contrib/legacy_seq2seq/python/ops/seq2seq.py", line 1092, in sequence_loss_by_example
crossent = softmax_loss_function(labels=target, logits=logit)
TypeError: sampled_loss() got an unexpected keyword argument 'logits'

hi 你这个demo 怎么安装？需要哪些依赖的软件？

hi 你这个demo 怎么安装？需要哪些依赖的软件？能把步骤写清楚一些么？

1.7.0的TensorFlow报参数的错

赵总：
1.7.0的TensorFlow，报的这个错有什么建议吗？
TypeError: sampled_loss() got an unexpected keyword argument 'logits'

完整log：
Traceback (most recent call last):
File "E:/PycharmProjects/chatbot-master_zhaoyingjun/chatbot-master/execute.py", line 259, in
train()
File "E:/PycharmProjects/chatbot-master_zhaoyingjun/chatbot-master/execute.py", line 124, in train
model = create_model(sess, False)
File "E:/PycharmProjects/chatbot-master_zhaoyingjun/chatbot-master/execute.py", line 94, in create_model
model = seq2seq_model.Seq2SeqModel( gConfig['enc_vocab_size'], gConfig['dec_vocab_size'], _buckets, gConfig['layer_size'], gConfig['num_layers'], gConfig['max_gradient_norm'], gConfig['batch_size'], gConfig['learning_rate'], gConfig['learning_rate_decay_factor'], forward_only=forward_only)
File "E:\PycharmProjects\chatbot-master_zhaoyingjun\chatbot-master\seq2seq_model.py", line 149, in init
softmax_loss_function=softmax_loss_function)
File "D:\ProgramData\Anaconda3\envs\tensorflow\lib\site-packages\tensorflow\contrib\legacy_seq2seq\python\ops\seq2seq.py", line 1224, in model_with_buckets
softmax_loss_function=softmax_loss_function))
File "D:\ProgramData\Anaconda3\envs\tensorflow\lib\site-packages\tensorflow\contrib\legacy_seq2seq\python\ops\seq2seq.py", line 1137, in sequence_loss
softmax_loss_function=softmax_loss_function))
File "D:\ProgramData\Anaconda3\envs\tensorflow\lib\site-packages\tensorflow\contrib\legacy_seq2seq\python\ops\seq2seq.py", line 1092, in sequence_loss_by_example
crossent = softmax_loss_function(labels=target, logits=logit)
TypeError: sampled_loss() got an unexpected keyword argument 'logits'

训练慢的问题

I7 CPU
训练好慢啊 用的小黄鸡语料
有什么解决办法么

seq2seq推理部分有问题

execute.py内decode_line函数会把词语转换成二进制的的值，这样在词字典中是找不到的，所以token_ids一直是[3,3,3,3,3,3……]
我把它替换成这样就行了：
import re
def decode_line(sess, model, enc_vocab, rev_dec_vocab, sentence):
# Get token-ids for the input sentence.
#token_ids = prepareData.sentence_to_token_ids(tf.compat.as_bytes(sentence), enc_vocab)
#token_ids = prepareData.sentence_to_token_ids(sentence, enc_vocab)
def sentence_to_token_ids(sentence, enc_vocab):
token_ids=[]
for word in re.split(r'\s+',sentence):
token_ids.append(enc_vocab.get(word,3))
return(token_ids)
token_ids=sentence_to_token_ids(sentence,enc_vocab)
bucket_id = min([b for b in xrange(len(_buckets)) if _buckets[b][0] > len(token_ids)])

下面的都不变

请问如何制作自己的语料库

想基于自己的FAQ材料，制作成适合咱们使用的语料库，请问使用何种工具可以制作？

不知道星星是哪来的

你这个代码，为什么训练的东西都是重复的一句话，per都到1了，别误人子弟了