alafalaki/compressedbart

The official implementation of...

compressedbart's Issues

Initialization from Bart-Large does not work

Hello,
Thank you for sharing the code for this project here.

I was trying to convert a Bart-large pre-trained checkpoint into CompressedBart. The checkpoint is available on HuggingFace. I used the following code:

ae = architectures.Autoencoder(bart_encoder_emb_size=768,
                            first_proj=first_projection,
                            second_proj=second_projection,
                            third_proj=third_projection,
                            max_len=max_input_length)
# Initialize compressedBART
cbart_config = BartConfig.from_pretrained(bart_checkpoint)
cbart_config.enc_d_model = cbart_config.d_model
cbart_config.d_model = 384
CBart_model = architectures.CBartForConditionalGeneration.from_pretrained(checkpoint_dir,
                                                           config=cbart_config,ae=ae,ignore_mismatched_sizes=True)

# Load the weights and put it on GPU
ae_checkpoint = os.path.join( checkpoint_dir, "ae-checkpoint.pth" )
if torch.cuda.is_available():
   ae_checkpoint = torch.load(ae_checkpoint)

else:
   ae_checkpoint = torch.load(ae_checkpoint, map_location=torch.device('cpu'))

CBart_model.model.encoder.ae.load_state_dict(ae_checkpoint['model'])
del ae_checkpoint

#CBart_model.eval()

if torch.cuda.is_available():
   CBart_model.to("cuda")

So far everything works. However, after I pass the model to the Trainer class and start evaluating (or training) I get the following error:

RuntimeError: mat1 and mat2 shapes cannot be multiplied (4096x1024 and 768x576)

Is it because I am using the 384 checkpoint and should instead use the 540? Could you please share this checkpoint again since the link expired.

Many thanks.
Best wishes.