About reproduce the results in paper.

[nbasyl]

About reproduce the results in paper.

[nbasyl]

Hi, Thanks for your interest, may I know which results are you referring to? Could you provide more details on " I find the results are pretty different for the same method in different paper."

[lucasliunju]

Hi, thanks for your reply.

I have reproduced the results of table 1.

For " I find the results are pretty different for the same method in different paper.", I found some paper uses a similar method (fine-tune llama with lora) but the results on the tasks of huggingface leaderboard are different. So I guess the codebase is important.

[nbasyl]

Yes, codebase is important as different prompt formats will often lead to different evaluation results.

[lucasliunju]

Hi, May I ask a question that is not related to your paper.

I woulk like to ask: What is the main problem of lora-based methods?

[nbasyl]

In my opinion, numerous difficulties persist, including overfitting and hyperparameters searching, compatibility with compression techniques, and many more issues that could be mentioned.