xuezhemax / flowseq Goto Github PK
View Code? Open in Web Editor NEWGenerative Flow based Sequence-to-Sequence Toolkit written in Python.
License: Apache License 2.0
Generative Flow based Sequence-to-Sequence Toolkit written in Python.
License: Apache License 2.0
Hi,
Thanks for sharing your paper,๏ผ
Can the methods in the paper be used to name entity recognition tasks?
I think the function ``bwdpass'' in file https://github.com/XuezheMax/flowseq/blob/master/flownmt/flows/flow.py is wrong. The line94 and line99 should be exchanged.
by the way, I have a question. What does bwdpass mean? maping a complex distribution(e.g. latent variables Z in your paper) to a simpler prior distribution (e.g. latent variables V in Eq.5 in your paper)?
finally, thanks for your great work and code release.
I used the script on fairseq to downlaod wmt14 data:
bash prepare-wmt14en2de.sh --icml17
But when I run the training program, it shows an error:
File "/content/flowseq/experiments/distributed.py", line 94, in run
single_process_main(args)
File "/content/flowseq/experiments/nmt.py", line 763, in main
train_iter, val_iter, test_iter = init_dataloader(args, dataset)
File "/content/flowseq/experiments/nmt.py", line 290, in init_dataloader
bucket_data=args.bucket_batch, multi_scale=args.length_unit)
File "/content/flowseq/flownmt/data/dataloader.py", line 291, in __init__
self.data_size, self.data = self.get_dataset()
File "/content/flowseq/flownmt/data/dataloader.py", line 337, in get_dataset
ftgt_ori = None if tgt_ori_path is None else codecs.open(tgt_ori_path, "r", encoding="utf-8")
File "/usr/lib/python3.6/codecs.py", line 897, in open
file = builtins.open(filename, mode, buffering)
FileNotFoundError: [Errno 2] No such file or directory: 'wmt14_en_de/tmp/dev.de.ori
Maybe I do not understand this paper throughly, but can someone explain this?
The posterior z
is modelled as diagonal Gaussian. And in the Zero initialization
part, ensures that the posterior distribution as a simple normal distribution
.
If it is a simple distribution, why a complex prior flow is needed to learn its distribution?
Thanks for the inspiring work!
When running the code I met a problem, maybe due to a version mismatch. I have to run in an environment with newer versions. Anyway, when I ran the python -u distributed.py
, I bumped into the error:
Traceback (most recent call last):
File "distributed.py", line 13, in <module>
from flownmt.data import NMTDataSet
File "/data/xjtupanda/experiments/audio_driven/head_motion/techs/flowseq-master/flownmt/__init__.py", line 1, in <module>
from flownmt.flownmt import FlowNMT
File "/data/xjtupanda/experiments/audio_driven/head_motion/techs/flowseq-master/flownmt/flownmt.py", line 11, in <module>
from flownmt.modules import Encoder
File "/data/xjtupanda/experiments/audio_driven/head_motion/techs/flowseq-master/flownmt/modules/__init__.py", line 1, in <module>
from flownmt.modules.encoders import *
File "/data/xjtupanda/experiments/audio_driven/head_motion/techs/flowseq-master/flownmt/modules/encoders/__init__.py", line 1, in <module>
from flownmt.modules.encoders.encoder import Encoder
File "/data/xjtupanda/experiments/audio_driven/head_motion/techs/flowseq-master/flownmt/modules/encoders/encoder.py", line 7, in <module>
class Encoder(nn.Module):
File "/data/xjtupanda/experiments/audio_driven/head_motion/techs/flowseq-master/flownmt/modules/encoders/encoder.py", line 25, in Encoder
def forward(self, src_sents, masks=None) -> Tuple[torch.Tensor, torch.Tensor]:
File "/data/xjtupanda/anaconda3/envs/gcn/lib/python3.7/site-packages/overrides/overrides.py", line 83, in overrides
return _overrides(method, check_signature, check_at_runtime)
File "/data/xjtupanda/anaconda3/envs/gcn/lib/python3.7/site-packages/overrides/overrides.py", line 170, in _overrides
_validate_method(method, super_class, check_signature)
File "/data/xjtupanda/anaconda3/envs/gcn/lib/python3.7/site-packages/overrides/overrides.py", line 189, in _validate_method
ensure_signature_is_compatible(super_method, method, is_static)
File "/data/xjtupanda/anaconda3/envs/gcn/lib/python3.7/site-packages/overrides/signature.py", line 102, in ensure_signature_is_compatible
ensure_return_type_compatibility(super_type_hints, sub_type_hints, method_name)
File "/data/xjtupanda/anaconda3/envs/gcn/lib/python3.7/site-packages/overrides/signature.py", line 303, in ensure_return_type_compatibility
f"{method_name}: return type `{sub_return}` is not a `{super_return}`."
TypeError: Encoder.forward: return type `typing.Tuple[torch.Tensor, torch.Tensor]` is not a `<class 'NoneType'>`.
Seems like it's caused by the behavior of @overrides
of package overrides
, I have no clue what this is doing, could you clarify this and make some modifications?
I'm runniing with Pytorh==1.10.1, Python==3.7.12, overrides==7.3.1. Thanks in advance.
Hello, I notice that your KL loss is calculated by "KL = (log_probs_posterior - log_probs_prior).mean(dim=1)". But I cannot understand why and cannot find a detailed explanation in your paper either. Could you help explain this to me or provide some references? Thanks so much!
Thanks for your excellent work.
Could you provide us with the trained model parameters and config file so that we can follow your work?
Hi, Thanks for sharing your code. How many steps or training time do it need to train the flowseq model on WMT14 EN-DE? Will you release the distillation dataset? It will be helpful for us to reproduce your results.
Why we need tgt_path_ori
and what is the difference between tgt_path_ori
and tgt_path
?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.