Thank you for your excellent work on this project! I have noticed that the repository includes a bash script for training FourierFT. Could you please share the training bash scripts for the baseline models, such as LoRA and AdaLoRA?
For example, how long does it take to finetune llama2-7B&13B with fourierft? Using the similar experimental setting, how long does it take to finetune llama2-7B&13B with lora?
Thank you very much for providing the open-source code.
I have a question about the initialization of F.
F, as a sparse matrix with a shape of (in_dim, out_dim), its non-zero indices are determined by E, where the values of E range from [0, in_dim*out_dim). This seems like it could cause index out-of-bounds errors.
Could you please tell me what details I might be missing?