zhihongshao/planning-based-hierarchical-variational-model

Dataset and code for ``Long and Diverse Text Generation with Planning-based Hierarchical Variational Model (EMNLP 2019)``

planning-based-hierarchical-variational-model's Issues

What's the source of the Chinese ad dataset

Hello,
I wonder how you build the Chinses ad dataset and what's the source of this dataset? Or it's done by crowdsource?
Thanks!

Xuming Lin

请教一下您使用的评测脚本可否提供呢？

比如distinct-4、repetition-4等。

a bug when i try to run preprocess.py

Traceback (most recent call last): File "preprocess.py", line 10, in <module> pickle.dump(Vocabulary.Vocabulary(), open(config.vocab_file, "wb")) File "/home/lzk/Planning-based-Hierarchical-Variational-Model/Vocabulary.py", line 33, in __init__ with open(self.config.wordvec_file) as file: FileNotFoundError: [Errno 2] No such file or directory: './data/processed/wordvec.txt'
how to fix it?

There was a problem training your own dataset

Hello, I have problems training my own dataset, category, feat_ key、feat_ The pkl files such as val have been changed and the data format has been adjusted according to yours, but there is still a problem. Can you help me solve this problem?
2022-09-25 21:29:19.479013: W tensorflow/core/framework/op_kernel.cc:1639] Unknown: KeyError: '类型'
Traceback (most recent call last):

File "/home/fanyongfeng/.conda/envs/fanyfeng/lib/python3.6/site-packages/tensorflow_core/python/ops/script_ops.py", line 235, in call
ret = func(*args)

File "/home/fanyongfeng/PyCharm/test_generation/Dataset.py", line 190, in
dataset = dataset.map(map_func=lambda x: tf.py_func(lambda y: process(y), [x], Tout=[tf.int32] * 14))

File "/home/fanyongfeng/PyCharm/test_generation/Dataset.py", line 186, in process
return self.process_inst(line)

File "/home/fanyongfeng/PyCharm/test_generation/Dataset.py", line 117, in process_inst
res["feats_key"].append(self.vocab.lookup(item[0], 0))

File "/home/fanyongfeng/PyCharm/test_generation/Vocabulary.py", line 85, in lookup
return self.table[tpe][word]

KeyError: '类型'

---

0 类型

---

Traceback (most recent call last):
File "/home/fanyongfeng/.conda/envs/fanyfeng/lib/python3.6/site-packages/tensorflow_core/python/client/session.py", line 1365, in _do_call
return fn(*args)
File "/home/fanyongfeng/.conda/envs/fanyfeng/lib/python3.6/site-packages/tensorflow_core/python/client/session.py", line 1350, in _run_fn
target_list, run_metadata)
File "/home/fanyongfeng/.conda/envs/fanyfeng/lib/python3.6/site-packages/tensorflow_core/python/client/session.py", line 1443, in _call_tf_sessionrun
run_metadata)
tensorflow.python.framework.errors_impl.UnknownError: {{function_node _inference_Dataset_map_453}} KeyError: '类型'
Traceback (most recent call last):

File "/home/fanyongfeng/.conda/envs/fanyfeng/lib/python3.6/site-packages/tensorflow_core/python/ops/script_ops.py", line 235, in call
ret = func(*args)

File "/home/fanyongfeng/PyCharm/test_generation/Dataset.py", line 190, in
dataset = dataset.map(map_func=lambda x: tf.py_func(lambda y: process(y), [x], Tout=[tf.int32] * 14))

File "/home/fanyongfeng/PyCharm/test_generation/Dataset.py", line 186, in process
return self.process_inst(line)

File "/home/fanyongfeng/PyCharm/test_generation/Dataset.py", line 117, in process_inst
res["feats_key"].append(self.vocab.lookup(item[0], 0))

File "/home/fanyongfeng/PyCharm/test_generation/Vocabulary.py", line 85, in lookup
return self.table[tpe][word]

KeyError: '类型'

 [[{{node PyFunc}}]]
 [[IteratorGetNext]]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "main.py", line 183, in
main()
File "main.py", line 161, in main
train(args.model_name, args.restore)
File "main.py", line 141, in train
_train(model_name, model, dataset, summary, init)
File "main.py", line 88, in _train
batchInput = dataset.next_batch(batch)
File "/home/fanyongfeng/PyCharm/test_generation/Dataset.py", line 66, in next_batch
res = self.input_sess.run(batch)
File "/home/fanyongfeng/.conda/envs/fanyfeng/lib/python3.6/site-packages/tensorflow_core/python/client/session.py", line 956, in run
run_metadata_ptr)
File "/home/fanyongfeng/.conda/envs/fanyfeng/lib/python3.6/site-packages/tensorflow_core/python/client/session.py", line 1180, in _run
feed_dict_tensor, options, run_metadata)
File "/home/fanyongfeng/.conda/envs/fanyfeng/lib/python3.6/site-packages/tensorflow_core/python/client/session.py", line 1359, in _do_run
run_metadata)
File "/home/fanyongfeng/.conda/envs/fanyfeng/lib/python3.6/site-packages/tensorflow_core/python/client/session.py", line 1384, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.UnknownError: {{function_node _inference_Dataset_map_453}} KeyError: '类型'
Traceback (most recent call last):

File "/home/fanyongfeng/.conda/envs/fanyfeng/lib/python3.6/site-packages/tensorflow_core/python/ops/script_ops.py", line 235, in call
ret = func(*args)

File "/home/fanyongfeng/PyCharm/test_generation/Dataset.py", line 190, in
dataset = dataset.map(map_func=lambda x: tf.py_func(lambda y: process(y), [x], Tout=[tf.int32] * 14))

File "/home/fanyongfeng/PyCharm/test_generation/Dataset.py", line 186, in process
return self.process_inst(line)

File "/home/fanyongfeng/PyCharm/test_generation/Dataset.py", line 117, in process_inst
res["feats_key"].append(self.vocab.lookup(item[0], 0))

File "/home/fanyongfeng/PyCharm/test_generation/Vocabulary.py", line 85, in lookup
return self.table[tpe][word]

KeyError: '类型'

 [[{{node PyFunc}}]]
 [[IteratorGetNext]]

数据集下载

数据集不能下载，链接有问题，能否换成其他网盘或者更新一下链接？

Could you provide scrip of processing data?

想用自己的数据集训练，作者可以提供您处理数据的脚本吗？
非常感谢。

Cant download dataset

Hi,
When I try to download the data from link, it failed.

dataset = Dataset.EPWDataset()

要改成这个把，train.py