alphadl/neural-graph-to-seq-mp

Code corresponding to our paper "A Graph-to-Sequence Model for AMR-to-Text Generation"

Neural Graph to Sequence Model

This repository contains the code for our paper A Graph-to-Sequence Model for AMR-to-Text Generation in ACL 2018

The code is developed under TensorFlow 1.4.1. We shared our pretrained model along this repository. Due to the compitibility reason of TensorFlow, it may not be loaded by some lower version (such as 1.0.0).

Please create issues if there are any questions! This can make things more tractable.

Data precrocessing

Our current data loading code requires simplified AMR graphs where variable tags, sense tags and quotes are removed. For example, the following AMR

(d / describe-01 :ARG0 (p / person :name (n / name :op1 "Ryan")) :ARG1 p :ARG2 genius)

can be simplified as

describe :arg0 ( person :name ( name :op1 ryan )  )  :arg1 person :arg2 genius

Our simplifier can be downloaded via here. It is generated by modifying the NeuralAMR code.

Here we briefly introduce how to use our simplifier. First make sure that each AMR takes exactly one line in your file. For example, you have a file 'demo.amr', which contains

(d / describe-01 :ARG0 (p / person :name (n / name :op1 "Ryan")) :ARG1 p :ARG2 genius)

Then, simply execute

./anonDeAnon_java.sh anonymizeAmrFull true demo.amr

and the program will generate file 'demo.amr.anonymized', which contains the simplified AMRs. Please note that our simplifier does not do anonymization, which is introduced by NeuralAMR (Konstas et al., 2017). The resulting file name has 'anonymized' because our simplifier is obtained by modifying NeuralAMR. We actually don't do anonymization.

Another alternative is to write your own data loading code according to the format of your own AMR data.

Input data format

The current input data format for our system is in JSON style demonstrated with the following sample:

[{"amr": "describe :arg0 ( person :name ( name :op1 ryan )  )  :arg1 person :arg2 genius",
"sent": "ryan's description of himself: a genius.",
"id": "demo"}]

In general, the JSON file contains a list of instances, and each instance is a dictionary with fields of "amr", "sent" and "id"(optional).

Training

First, modify the PYTHONPATH within train_g2s.sh (for our graph-to-string model) or train_s2s.sh (for baseline).
Second, modify config_g2s.json or config_s2s.json. You should pay attention to the field "suffix", which is an identifier of the model being trained and saved. We usually use the experiment setting, such as "bch20_lr1e3_l21e3", as the identifier.
Finally, execute the corresponding script file, such as "./train_g2s.sh".

Using large-scale automatic AMRs

In this setting, we follow Konstas et al., (2017) to take the large-scale automatic data as the training set, with the original gold data being the finetune set. To train in this way, you need add a new field "finetune_path" in the configuration file and point it to the gold data. Then, point "train_path" to the automatic data.

For training on the gold data only, we use an initial learning rate of 1e-3 and L2 normalization of 1e-3. We then lower the learning rate to be 8e-4, 5e-4 and 2e-4 after a number of epoches.

For training on both gold and automatic data, the initial learning rate and L2 normalization are 5e-4 and 1e-8. We also lower the learning rate during training.

The idea of lowering learning rate was first introduced by Konstas et al., (2017).

Decoding with a pretained model

Simply execute the corresponding decoding script with one argument being the identifier of the model you want to use. For instance, you can execute "./decode_g2s.sh bch20_lr1e3_l21e3"

Pretrained model

We releaes our recently pretrained model using gold plus 2M automatically-parsed AMRs here. With this model, we observed a BLEU of 33.6, which is higher than our paper-reported number of 33.0.

Cite

If you like our paper, please cite

@InProceedings{song-EtAl:acl2018,
  author    = {Song, Linfeng  and  Zhang, Yue  and  Wang, Zhiguo  and  Gildea, Daniel},
  title     = {A Graph-to-Sequence Model for {AMR}-to-Text Generation},
  booktitle = {Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (ACL-18)},
  year      = {2018},
  address   = {Melbourne, Australia},
}