fastnlp/style-transformer

Style Transformer: Unpaired Text Style Transfer without Disentangled Latent Representation

This folder contains the code for the paper 《Style Transformer: Unpaired Text Style Transfer without Disentangled Latent Representation》

Requirements

pytorch >= 0.4.0

torchtext >= 0.4.0

nltk

fasttext == 0.8.3

kenlm

Usage

The hyperparameters for the Style Transformer can be found in ''main.py''.

The most of them are listed below:

    data_path : the path of the datasets
    log_dir : where to save the logging info
    save_path = where to save the checkpoing
    
    discriminator_method : the type of discriminator ('Multi' or 'Cond')
    min_freq : the minimun frequency for building vocabulary
    max_length : the maximun sentence length 
    embed_size : the dimention of the token embedding
    d_model : the dimention of Transformer d_model parameter
    h : the number of Transformer attention head
    num_layers : the number of Transformer layer
    batch_size : the training batch size
    lr_F : the learning rate for the Style Transformer
    lr_D : the learning rate for the discriminator
    L2 : the L2 norm regularization factor
    iter_D : the number of the discriminator update step pre training interation
    iter_F : the number of the Style Transformer update step pre training interation
    dropout : the dropout factor for the whole model

    log_steps : the number of steps to log model info
    eval_steps : the number of steps to evaluate model info

    slf_factor : the weight factor for the self reconstruction loss
    cyc_factor : the weight factor for the cycle reconstruction loss
    adv_factor : the weight factor for the style controlling loss

You can adjust them in the Config class from the ''main.py''.

If you want to run the model, use the command:

python main.py

To evaluation the model, we used Fasttext, NLTK and KenLM toolkit to evaluate the style control, content preservation and fluency respectively. The evaluation related files for the Yelp dataset are placed in the ''evaluator'' folder.

Because the file "ppl_yelp.binary" is too big to upload, we exclude it from the "evaluator" folder. As a result, you can not evaluate the ppl score via evaluator. To solve this problem, you can use the KenLM toolkit to train a language model by yourself or use other script to evaluate it.

Outputs

Update: You can find the outputs of our model in the "outputs" folder.

style-transformer's Issues

fasttext 0.8.3

my fasttext version is 0.9.2, the error is
ValueError : acc_yelp.bin has wrong file format

please help me

"ppl" metrics is different from the paper

Hi, I run the code to acquire the value of "ppl" metrics about 150, may I ask how you calculated the 70 in your paper, thank you very much.

ACC Wrong

Hello, I trained acc_yelp.bin on fasttext0.9.2, but when training the model, the accuracy rate on pos was always 1, while the neg sample was always 0. I want to know if you trained acc_yelp
.bin step

FileNotFoundError: No such file or directory: '/style-transformer-master/evaluator/ppl yelp.binary'. Does anyone have this problem?

FileNotFoundError: No such file or directory: '/style-transformer-master/evaluator/ppl yelp.binary'. Does anyone have this problem?

使用预训练embedding bleu偏低

您好，为什么使用预训练词向量和不使用预训练词向量，会有一些差距，感觉使用词向量的时候bleu会特别低，对于yelp数据集

Potential bug in train.py

The code in Line 205 in file train.py "adv_log_porbs = model_D(gen_soft_tokens, gen_lengths, rev_styles)" should be in the ELSE? Because it uses style information in classfication and should be "Cond" method?

Is it okay about the abscent of forward in self.embed in StyleTransformer.forward?

Hi I'm Jaehee from South Korea
I'm studying your paper recently.
while I reviewed your codes, I think there must be forward method in self.embed in StyleTransformer.forward of 50th line of Transformer.py

Is it right that you were meant self.embed to forward?

the code is below
enc_input = torch.cat((style_emb, self.embed(inp_tokens, pos_idx[:, :max_enc_len])), 1)

ValueError: E:\style-transformer\evaluator\acc_yelp.bin has wrong file format!

hi!
Thanks for your code and paper, I get a lot from it.
But when I run the code,I met the error ValueError: E:\style-transformer\evaluator\acc_yelp.bin has wrong file format! in the line self.classifier_yelp = fasttext.load_model(yelp_acc_file.name) , I know that you don't provide a file named ppl_yelp.binary, I just commented related code.
I think maybe the file acc_yelp.bin has some format error? But I am not sure, looking forward to your reply, thanks!

fasttext == 0.8.3

Hao install fasttext==0.8.3

evaluator文件夹里的ppl_yelp.binary和acc_yelp.bin

您好，
在运行的时候，发生了错误，evaluator/acc_yelp.bin has wrong file format!

然后，您能不能更详细地说明一下如何生成这两个文件，非常感谢！！！

About multi-class dataset

Hi, I want to train a model on my dataset, which has 5 classes. I feel confused about the data preprocessing about multi-classes dataset. Any codes or descriptions about it in this github? Thanks very much!

Add License

Hey,

I would like to use your code and build up on it.
Could you please add a license to your repository, that allows me to use it for research?
I see, the fastNLP repository already has a license.

Thanks.

请问要大约多少iters可以达到最好的结果

您好，
非常感谢您的分享！我在运行的过程中，发现代码在循环里一直反复进行iter的训练，请问大概在多少iters的时候可以达到相对最优的结果呢，或者论文中的结果大概需要运行多少iters(文中列出的结果应该是对pos和neg各个指标的均值吧)~
谢谢！

i can't find the file of "ppl_yelp.binary"