yg211/acl20-ref-free-eval

SUPERT: Unsupervised multi-document summarization evaluation & generation

acl20-ref-free-eval's Issues

About instructions of reproducing Table 2, 4, and 5 in the paper

Thank you for sharing this nice work!

Could you provide some instructions to reproduce the results of Table 2, 4, and 5 in the paper? Thanks again.

Summary length

Hello.
I'd like to know if it's possible to adjust length of output summary?

Model.encode() error

Problem:

When I run the sample code provided for evaluation, I run for the following error in the file supert.py:

Checking the SBERT documentation, I see that model.encode() only returns embeddings, not the tokens itself.

I tried to use the AutoTokenizer from huggingface in order to return tokens, however I ran into assertion error as length of tokens were not equal to length of embeddings.

Before doing more reverse engineering, I wanted to ask you what you wanted to achieve here and if it is a deprecated usage how can we solve it?

Thank you for your help!

Running the Summarizer

Hi, I'm relatively new to using GitHub and Python... How can I actually run the summarizer on my documents?

I am receiving an error that python can't find "ref_free_metrics" or "generate_summary_rl" when I try to run the code in the readme.

Thanks!

UnboundLocalError: local variable 'full_token' referenced before assignment

This error occurs when running evaluate_summary.py without the optionnal part for one file with 3 summaries (data/input_docs/file_1.txt and data/summaries/summary_1.txt, data/summaries/summary_2.txt, data/summaries/summary_3.txt). My text is smaller than 15 sentences.

GPU usage

Hi. Is it possible to train model on GPU?

The "summariser/utils" folder is missing in the compare_metrics branch

Dear Yang,
It seems the "summariser/utils" folder is missing in the compare_metrics branch. I am trying to use the "utils" folder in the master branch as a succedaneum. But there may have some differences. Could you upload it? Thank you.