Usage of the 'masks' in hf_data

[lhao499]

Usage of the 'masks' in hf_data

[lhao499]

Thanks for catching this, we made an mistake while releasing it, pushed an update.

[g131002]

Thank you for the quick fix!
So you actually used 'masks' part in the experiment, right?
Also, I checked the part of the code where you fixed, and don't you also have to take care of the bos_token part?
And about the bos_token part, did you also append the bos_token when you did the evaluation?

[lhao499]

Yeah, using masks is important, otherwise model will generate 'user' part which is undesired.

[lhao499]

bos_token is for input, loss_mask will take care of output and loss. bos_token is not used in evaluation because we have question/instruction as input and not doing unconditional generation.