Comments (4)
我这边用的是标贝数据集,大概6k步的时候就能学到很好的对齐
from tacotronv2_wavernn_chinese.
感谢你的回复,是我自己实现的问题,我已经修复了这个bug. 我有一点好奇的是。
我对你的合成的对对齐图额外处理操作感到一点困惑,如果你不介意的话,能在有空的时候做一些解释么,十分感谢?
如果不做这个操作就不能生成很好的效果么?
if not self.is_training: # prevent repeat and stay too long
print('' * 100)
print('calling the part.')
print('' * 100)
Tx = tf.shape(shift_alpha)[1]
max_attentions = tf.where(tf.less_equal(max_attentions, state.max_attentions),
state.max_attentions, state.max_attentions+1)
short_thres = tf.ones_like(state.pos_rec, dtype=tf.int32) * 5
short_val = tf.ones_like(max_attentions) * 2
short_mask = tf.logical_and(tf.less(state.pos_rec, short_thres),
tf.less(short_val, max_attentions))
max_attentions = tf.where(short_mask, state.max_attentions, max_attentions)
pos_mask = tf.equal(max_attentions, state.max_attentions)
ones_val = tf.ones_like(pos_mask, dtype=tf.int32)
pos_rec = tf.where(pos_mask, state.pos_rec + 1, ones_val)
thres = tf.ones_like(state.pos_rec, dtype=tf.int32) * 10
pos_mask = tf.less(pos_rec, thres)
max_attentions = tf.where(pos_mask, max_attentions, max_attentions+1)
pos_rec = tf.where(pos_mask, pos_rec, ones_val)
left = tf.sequence_mask(max_attentions-2, Tx)
right = tf.logical_not(tf.sequence_mask(max_attentions+3, Tx))
mask = tf.logical_not(tf.logical_or(left, right))
paddings = tf.zeros_like(shift_alpha)
alignments = tf.where(mask, alignments, paddings)
left = tf.sequence_mask(tf.clip_by_value(max_attentions, 0, Tx-1), Tx)
right = tf.logical_not(tf.sequence_mask(max_attentions+1, Tx))
mask = tf.logical_not(tf.logical_or(left, right))
max_alignments_values = tf.reduce_sum(alignments, axis=-1, keepdims=True)
max_alignments_values = tf.where(tf.less(max_alignments_values,
tf.ones_like(max_alignments_values, dtype=tf.float32) * 1e-10),
tf.ones_like(max_alignments_values, dtype=tf.float32),
max_alignments_values)
alignments = tf.where(mask, tf.zeros_like(alignments) + max_alignments_values * 2.0, alignments)
from tacotronv2_wavernn_chinese.
对对齐图额外处理操作主要是让模型在inference阶段(训练阶段,这个操作不执行)能合成长文本,以及避免重复读(短文本中也存在)的问题。
这个操作主要了控制每个音素的时长范围(5 - 10)。如果不执行这段操作,也能正常合成的wav,合成的效果感觉比执行这段合成的要好。当时实现的时候参考这里,由于pytorch跟tensorflow的差异,加上自己的修改,最终的实现方式不太一样
from tacotronv2_wavernn_chinese.
十分感谢
from tacotronv2_wavernn_chinese.
Related Issues (20)
- 如何增加symbol? HOT 1
- 推理多路并发 HOT 2
- 关于使用tacotron2训练过拟合的问题,wavernn调用的问题 HOT 1
- 关于wavernn preprocess 的语料准备 HOT 2
- 分享一下音色finetune的结果 HOT 3
- 如何加载 gmm attention HOT 1
- 尝试生成训练数据,运行python tacotron_preprocess.py HOT 1
- 无法使用gpu训练 HOT 1
- speaker adaptive 微调参数 HOT 1
- 您好,请问有测试过合成声音的实时性吗? HOT 3
- finetune训练速度比较慢 HOT 1
- 关于“align.png”问题 HOT 1
- 对齐图错位问题 HOT 3
- 请问预测阶段的batch_size如何调大,以及对应的预测数据如何调整呢,谢谢 HOT 1
- where is the docker images? HOT 1
- tacotron_synthesize.py 生成梅尔频谱需要两秒到三秒,请问Tacotron2是不是比较慢,还是其他调参的原因? HOT 1
- 关于graves attention如何修改architecture_wrappers.py的问题 HOT 2
- 生成不完整 HOT 1
- 关于路径配置问题 HOT 3
- 请教下,怎么生成16000 Hz的wav? HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from tacotronv2_wavernn_chinese.