benjaminocampo/gen_hs_explanations

Hello I'm modifying develop

gen_hs_explanations's Issues

Add generation files to the repository

Preprocess column and generation files

Differenciate finetuning through api and finetuning through huggingface

Error Analysis on HateCheck

Description

Analyze the results obtained with HateCheck and the models GPT3, GPT3.5 and GPT4.

Steps

• Merge the metrics BERTScore and BLEURT using Acc@s.
• Highlight which functionalities perform well in terms of Acc@s.
• Choose interesting examples that are well and badly generated to highlight.

Check what happened with Alpaca

Send an email to Meta?

Make an script to deal with null values

Deprecate old scripts for finetuning

Adapt the reported results spreadsheets and link it to the repository.

Update functionality definitions: Change gold_explanation in the HateCheck dataset.

Functionality	Old Explanation	New Gold Explanation
F1	Expression of strong negative emotions (Explicit)	Expression of strong negative emotions promoting prejudice and discrimination.
F2	Description using very negative attributes (Explicit)	Description using very negative attributes promoting prejudice and discrimination.
F4	Implicit Derogation	Check them manually?
F12	Hate expressed through reference in subsequent clauses	Can we say that it is the same as F1 or F2?
F13	Hate expressed through reference in subsequent sentences	Can we say that it is the same as F1 or F2?
F8	Non-hateful homonyms of Slurs	Non-hateful homonyms of Slurs, in this case, [HOMONYM]

Move scripts for annotation agreement in the experiments folder

Update T5 Results on the Spreadsheets with BLEU, BERT, and BLEURT Scores

Deadline Meeting on Wednesday 25.

Deprecate old scripts for few-shot

Update REV scores with TF-IDF

Migrate notebooks to the repo

Description

Migrate notebooks we used for generating messages and obtain results to a new directory called notebooks.

Steps

• Migrate those notebooks and check they are runnable with the project's environment. Add any missing libraries to environment.yml

Finetune T5 and GPTNeoX

Description

Finetune T5 and GPTNeoX using HateCheck data.

Steps

• Experiment fine-tuning already pre-trained T5 tasks such as "summarize:" but using hateful data.
• Try other possible prompts.
• Save models in huggingface so we can use them later.

Write README.md including installation, structure, how to contribute, etc.

Remove unused csv files. Use only those reported in the paper spreadsheets.