tatianapassali/artificial-disfluency-generation

Generating artificial disfluencies from fluent text easily and promptly

artificial-disfluency-generation's Issues

bug related to splits in create_dataset.py

Hi!

I found a bug related to splits in create_dataset.py and I wrote a pull request to fix the bug and a few other minor issues i found on the way.

Could you check it and merge if it is everything alright?

thanks

LARD does not work well on switchboard datasets

Hi Passali! I did some experiments using the method "Lard" and would like to ask some questions.
I did three experiments:

1. finetune BERT using the synthetic dataset you provided from Zenodo to do disfluency detection(classify each token of the sequence as fluent or disfluent)
2. remove disfluency in Switchboard to get fluent text firstly, then use Lard to generate disfluency, finally finetune BERT using the synthetic dataset to do disfluency detection
3. finetune BERT with Switchboard dataset

results are followed:

It is observed that synthetic data with Lard did badly on benchmark Switchboard, is it because the disfluencies generated are
not realistic?