daveshap/gpt3_finetunes

Public repo for my finetuning projects

GPT3_Finetunes

Public repo for my finetuning projects. Check each folder for a finetuning file and description. Work in progress.

gpt3_finetunes's Issues

Question: How do you decided when the prompt end and where completion begins?

Hello David,

I came here from your video 5 Tips and Misconceptions about Finetuning GPT-3

You mentioned there to use Natural Language Separators. But in the chatbot example, the separator seems incoherent. Some completions are the last part of the conversation, some are in the middle. What is the reason for that and how does it make the results better?

The reason I'm asking is I'm building a bot that initiates the conversation with some information upfront (as opposed to the user starting out with a question). The bot is the driver of the conversion from beginning to end. What will be a good fine-tuning strategy or prompt engineering to accomplish that?

Thank you!