buhu2k/awesome-personalized-llm

This repo aims to record resource of role-playing abilities in LLMs, including dataset, paper, application, etc.

Awesome-Personalized-LLM

This repo aims to record resource of personalized LLMs, including role playing, personalized chat, and personality traits of LLm etc.

We strongly encourage the researchers that want to promote their fantastic work to the personalized LLM community to make pull request to update their paper's information!

• Awesome-Personalized-LLM
  • Awesome Papers
    • Role-Playing
    • Personalized Chat
    • Evaluation Personality of LLM
      • Myers–Briggs Type Indicator (MBTI) Evaluation
      • Big Five Factors Evaluation
      • Other Personal Evaluation
    • Benchmark
  • Awesome Datasets
  • Related Repositories

Awesome Papers

Role-Playing

Title	Venue	Date	Code	Demo
RoleInteract: Evaluating the Social Interaction of Role-Playing Agents	Arxiv24	2024-03-21	RoleInteract	-
Unveiling the Secrets of Engaging Conversations: Factors that Keep Users Hooked on Role-Playing Dialog Agents	Arxiv24	2024-03-13	-	-
Neeko: Leveraging Dynamic LoRA for Efficient Multi-Character Role-Playing Agent	Arxiv24	2024-02-21	Neeko	-
Enhancing Role-playing Systems through Aggressive Queries: Evaluation and Improvement	Arxiv24	2024-02-16	-	-
Large Language Models are Superpositions of All Characters: Attaining Arbitrary Role-play via Self-Alignment	Arxiv24	2024-01-24	Ditto	-
CharacterGLM: Customizing Chinese Conversational AI Characters with Large Language Models	Arxiv23	2023-11-19	CharacterGLM-6B	-
Character-LLM: A Trainable Agent for Role-Playing	EMNLP23	2023-10-16	trainable-agents	-
Large Language Models Meet Harry Potter: A Bilingual Dataset for Aligning Dialogue Agents with Characters	EMNLP23	2023-10-09	HPD	-
NarrativePlay: Interactive Narrative Understanding	arXiv	2023-10-02	-	-
RoleLLM: Benchmarking, Eliciting, and Enhancing Role-Playing Abilities of Large Language Models	arXiv	2023-10-01	RoleLLM	-
ChatHaruhi: Reviving Anime Character in Reality via Large Language Model	arXiv	2023-08-18	ChatHaruhi	ChatHaruhi-OpenAI
Thespian: Multi-Character Text Role-Playing Game Agents	arXiv	2023-08-03	-	-
Role-Play with Large Language Models	arXiv	2023-05-25	-	-
Ontologically Faithful Generation of Non-Player Character Dialogues	arXiv	2023-05-13	-	-

Personalized Chat

Title	Venue	Date	Code	Demo
ControlLM: Crafting Diverse Personalities for Language Models	Arxiv	2024-02-15	ControlLM	-
Tailoring Personality Traits in Large Language Models via Unsupervisedly-Built Personalized Lexicons	Arxiv	2023-10-25	-	-
Personalized Soups: Personalized Large Language Model Alignment via Post-hoc Parameter Merging	Arxiv	2023-10-17	RLPHF	-
Large Language Models as Source Planner for Personalized Knowledge-grounded Dialogues	Arxiv	2023-10-15	-	-
CharacterChat：Learning towards Conversational AI with Personalized Social Support	arXiv	2023-08-20	CharacterChat	-
Teach LLMs to Personalize – An Approach inspired by Writing Education	Arxiv	2023-08-15	-	-
A Survey of Challenges and Methods in the Computational Modeling of Multi-Party Dialog	NLP4ConvAI23	2023-07-14	-	-
Multi-Party Chat: Conversational Agents in Group Settings with Humans and Models	Arxiv	2023-06-08	MultiLIGHT	-
Hi Sheldon! Creating Deep Personalized Characters from TV Shows	arXiv	2023-04-09	Deep-Personalized-Character-Dataset-DPCD	-
Role Specified Open-Domain Dialogue dataset generated by leveraging LLMs	NAACL22	2022-04-30	CareCall for Seniors	-

Evaluation Personality of LLM

Myers–Briggs Type Indicator (MBTI) Evaluation

Title	Venue	Date	Code	Demo
Does Role-Playing Chatbots Capture the Character Personalities? Assessing Personality Traits for Role-Playing Chatbots	Arxiv23	2023-10-27	Chat-Haruhi-Suzumiya	-
Can ChatGPT Assess Human Personalities? A General Evaluation Framework	EMNLP23	2023-10-13	ChatGPT-MBTI	-
Do LLMs Possess a Personality? Making the MBTI Test an Amazing Evaluation for Large Language Models	Arxiv	2023-07-30	LLMs-MBTI	-

Big Five Factors Evaluation

Title	Venue	Date	Code	Demo
Evaluating and Inducing Personality in Pre-trained Language Models	NeurIPS23	2023-05-24	-	-
Estimating the Personality of White-Box Language Models	Arxiv	2023-05-10	-	-
Identifying and Manipulating the Personality Traits of Language Models	Arxiv	2022-12-20	-	-

Other Personal Evaluation

Title	Venue	Date	Code	Demo
Do personality tests generalize to Large Language Models?	NeurIPS23-Workshop	2023-11-10	-	-
The Turing Quest: Can Transformers Make Good NPCs?	ACL23-Workshop	2023-07-10	NPC-Dialogue-Generation	-
Have Large Language Models Developed a Personality?: Applicability of Self-Assessment Tests in Measuring Personality in LLMs	Arxiv	2023-05-24	-	-
Who is GPT-3? An Exploration of Personality, Values and Demographics	Arxiv	2022-12-26	-	-

Benchmark

Title	Venue	Date	Code	Demo
CharacterEval: A Chinese Benchmark for Role-Playing Conversational Agent Evaluation	Arxiv24	2024-01-02	CharacterEval	-
RoleEval: A Bilingual Role Evaluation Benchmark for Large Language Models	Arxiv23	2023-12-12	RoleEval	-
LaMP: When Large Language Models Meet Personalization	Arxiv23	2023-05-19	-	-

Awesome Datasets

• GPTeacher/Roleplay
• BelleGroup/Generated_Chat_0.4M
• CharacterChat
• Deep-Personalized-Character-Dataset-DPCD
• Harry Potter Dialogue
• carecall-corpus (Korean)
• RoleBech

Related Repositories

• Paper reading list in dialogue systems
• LLM and Society
• Persona Paper