asappresearch/flop

Pytorch library for factorized L0-based pruning.

flop's Issues

Is this method applicable to LLAMA?

Where is the code for section 4.2?

Hi,

I am looking for the implementation of your matrix factorization for the key, value and FFNN matrices in Fine-tuning BERT on classification tasks. Have you provided the code of modified BERT with your matrix factorization?

Thanks.

Implementation of fine-tuning RoBERTa on downstream tasks

Hi,

Thank you so much for publishing your code for the paper! However, I only found the implementation of Task 1 (recurrent word-level language models on Wiki-103) and Task 2 (recurrent character-level language models on Enwik8) described in the Experimental Setup (Section 5) of the paper.

Do you have the code for Task 4, namely fine-tuning RoBERTa on downstream tasks, available? I'm very interested in reproducing that part of the experiment and would really appreciate it if you could provide the implementation.