RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn

[KaifAhmad1]

RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn

[RonanKMcGovern]

I'm reasonably confident this is because SFTTrainer seems to have changed how it expects args to be passed.

SFT_config must be used.

Also, it seems that peft_config cannot be passed within SFT_config. See here.

It took me a while to track this down but it appeared in an error message.

Kind of annoying if it is this, because a lot of my SFT scripts have now broken (and I didn't have frozen versioning because I was using the dev version of SFT in quite a few cases).

[amyeroberts]

Hi @KaifAhmad1, in order to be able to help we need a full reproducer i.e. something we can copy, paste and run to get the same error. Without the model and dataset (or public equivalents) or peft config there's not much we can do here

cc @younesbelkada @pacman100

[KaifAhmad1]

Hey, @amyeroberts I cannot share the full code here. Here is the link of colab notebook so you can refer.
https://github.com/KaifAhmad1/code-test/blob/main/Phi_3_Fine_Tuned_on_Indic_Lanuage.ipynb

[younesbelkada]

Hi @KaifAhmad1
Can you try to force-set gradient_checkpointing:

# Training Arguments
training_arguments = TrainingArguments(
    output_dir="./results",
    num_train_epochs=1,
    per_device_train_batch_size=4,
    gradient_accumulation_steps=1,
    optim="paged_adamw_32bit",
    save_steps=0,
    logging_steps=25,
    learning_rate=2e-4,
    weight_decay=0.001,
    fp16=True,
    bf16=False,
    max_grad_norm=0.3,
    max_steps=-1,
    warmup_ratio=0.03,
    group_by_length=True,
    lr_scheduler_type="cosine",
    report_to="tensorboard",
+   gradient_checkpointing=True
)
     

# SFTTrainer Arguments
trainer = SFTTrainer(
    model=model,
    train_dataset=train_dataset,
    peft_config=peft_config,
    dataset_text_field='text',
    args=training_arguments,
    tokenizer=tokenizer,
    packing=False,
    max_seq_length=None
)
   

[KaifAhmad1]

Thanks, @younesbelkada for fixing it!

[younesbelkada]

Thanks @RonanKMcGovern for the feedback, indeed the SFTConfig might have broken things .. I will look into that, can you help me providing more details here on what are the silent bugs you faced ? In our CI where we extensively test many things, all seemed green for us, this will help improve our testing infra
Can you elaborate on why peft_config cannot be passed within SFTConfig ?