skywalker023/sodaverse

🥤🧑🏻‍🚀Code and dataset for our EMNLP 2023 paper - "SODA: Million-scale Dialogue Distillation with Social Commonsense Contextualization"

Welcome to SODAverse 🌟

This is the official repository for our EMNLP paper:
SODA: Million-scale Dialogue Distillation with Social Commonsense Contextualization

🏆 SODA won the Outstanding Paper Award at EMNLP 2023!

@inproceedings{kim2023soda,
    title={SODA: Million-scale Dialogue Distillation with Social Commonsense Contextualization},
    author={Hyunwoo Kim and Jack Hessel and Liwei Jiang and Peter West and Ximing Lu and Youngjae Yu and Pei Zhou and Ronan Le Bras and Malihe Alikhani and Gunhee Kim and Maarten Sap and Yejin Choi},
    booktitle={EMNLP},
    year={2023}
}

For a brief summary of our paper, please see this tweet.

🫧 CO₃

Environment setup

We recommend you create a conda environment as follows:

conda env create -f environment.yml

and activate it with

conda activate sodaverse

Running CO₃

python co3.py --display-frequency 1 --model gpt-3.5-turbo-1106 --run-id first-run-with-chatgpt

This command will automatically download Atomic10x and start running the CO₃ distillation to make SODA. All files will be saved under the data directory, which will also be created during the run.

Running on other large language models (LLMs)

Currently, this repository only supports GPT models. However, you can add other LLMs to the agents directory. All you need to do is create an agent class with the method interact().

🥤SODA

You can now load SODA from the HuggingFace hub as the following:

from datasets import load_dataset

dataset = load_dataset("allenai/soda")

🧑🏻‍🚀COSMO

You can now load COSMO-3B from the HuggingFace hub.

Have a chat with Cosmo!

🚨 Disclaimer: We would like to emphasize that COSMO is trained on SODA and ProsocialDialog mainly for academic/research purposes. We discourage using COSMO in real-world applications or services as is. Model outputs should not be used for advice for humans, and could be potentially offensive, problematic, or harmful. The model’s output does not necessarily reflect the views and opinions of the authors and their associated affiliations.

Running Cosmo

By running the command below, Cosmo will be automatically downloaded and you can have a chat!

python chat_with_cosmo.py

Limitations

Cosmo is mostly trained on social chitchat. Therefore, we do not encourage having knowledge-intensive conversations (e.g., science, medical issues, law). For detailed limitations, please refer to our paper.

sodaverse's Issues

Fine tune ourselves?

Hi, is it currently possible to take your pretrained model cosmo-xl and fine tune it using conversation text from a certain domain, so that this new set of knowledge can be used during dialogue with cosmo?

Python/MacOSX & Protocol Buffers

I was missing the binary and got that installed, installed the same (protocol buffers) with pip and I still get this when I tried to run it on OSX:

If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.
If you cannot immediately regenerate your protos, some other possible workarounds are:
 1. Downgrade the protobuf package to 3.20.x or lower.
 2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).

More information: https://developers.google.com/protocol-buffers/docs/news/2022-05-06#python-updates

Will you add a bigger model than 3B?

would you release the 11B model?

thanks.

Add search-server as fallback for dialogue

Can you add search-server as fallback for dialogue like BlenderBot2?

Example
1 => "What are the latest news in Denmark"?
2 => AI check, if 1 => search else 0 => no search
3 => search = 1 => Search "lastest news in denmark" => Get text => Send text to sodaverse and sodaverse will rewrite text for the dialogue.

It will work good with search-server as fallback for sodaverse

Examples
https://github.com/JulesGM/ParlAI_SearchEngine
https://www.blenderbot.ai (Blenderbot3 with search-server, you need VPN or to be located in USA to test)

Are you planning to release the source code that you used to train sodaverse?

I am looking to add more data to the model and other modifications.

Can you add examples for "situation", "instruction", "dialogue_history" like https://github.com/f/awesome-chatgpt-prompts ?

Do you have more examples for "situation", "instruction", "dialogue_history"?
Like https://github.com/f/awesome-chatgpt-prompts

situation_narrative = "Cosmo is having a friendly conversation with a friend."
role_instruction = "You are Cosmo and you are talking to a friend."
dialogue_first = "Hey, how was your trip to Abu Dhabi?"

I tried
situation_narrative = "{{avatarName}} is having a friendly conversation with a friend."
role_instruction = "You are {{avatarName}} and you are talking to a {{humanName}}."
dialogue_first_avatar = "My name is {{avatarName}}."
dialogue_first_human = "My name is {{humanName}}."

Is there a better setup? I can't get "What is my name?" to work. Cosmo-xl thinks that I am Bill Gates. I am not Bill Gates. I am chatting with Bill Gates on www.justlearn.com

Code for data set gen?

Are you planning to release this code? I think your paper calls this CO2…