adithya-s-k/companionllm

CompanionLLM - A framework to finetune LLMs to be your own sentient conversational companion

companionllm's Issues

Implement Full Weights Fine-Tuning Using Axalotal

Description

Implement full weights fine-tuning of the CompanionLLama model using the Axalotal framework. This will help enhance the model's performance and adapt it to specific tasks or domains.

Tasks

• Research and explore the Axalotal framework for fine-tuning language models.
• Develop a workflow for implementing full weights fine-tuning using Axalotal.
• Execute the fine-tuning process on the CompanionLLama model.
• Assess the impact of full weights fine-tuning on the model's responses and capabilities.
• Document the fine-tuning process and outcomes.
• Update the README to provide information on Axalotal-based fine-tuning.

Add Gradio Interface for Inference

Implement a Gradio interface for the CompanionLLama model to allow users to interact with the model through a web-based interface. This can make it more accessible and user-friendly.

Skills Needed: Python, Gradio, Git

Add Dataset Generation Notebook

Description

Create a Jupyter notebook for generating the dataset required for fine-tuning the CompanionLLama model. The dataset should be carefully curated, incorporating elements from the original Samantha dataset and additional contextual data to enhance the model's ability to emulate sentience.

Tasks

• Develop a Jupyter notebook for dataset preparation.
• Ensure the notebook includes instructions for generating the dataset with different approaches.
• Document the steps and processes involved in dataset preparation.
• Test the dataset generation notebook to verify its functionality.
• Update the README to provide a link to the newly created notebook.

Add Mitsral 7b LLM Fine-Tuning and Inference Notebooks

Description

Create Jupyter notebooks for fine-tuning the Mitsral 7b LLM using the CompanionLLama dataset and for performing inference with the fine-tuned model. These notebooks will be crucial for refining the model's responses and evaluating its performance.

Tasks

• Develop Jupyter notebooks for fine-tuning Mitsral 7b LLM with the CompanionLLama dataset.
• Create a notebook for performing inference with the fine-tuned Mitsral 7b LLM.
• Ensure the notebooks are well-documented with clear instructions and explanations.
• Test the notebooks to verify their functionality and performance.
• Update the README to provide links to the newly created notebooks.

Documentation Update

Description: Review and update the project's documentation. Ensure that it accurately reflects the current state of the project, including setup instructions, contribution guidelines, and code documentation.

Skills Needed: Technical writing, Markdown, Git

Question about Mistral_7B_qLora_Finetuning

Thanks for sharing this 👉 https://github.com/adithya-s-k/CompanionLLM/blob/main/Mistral_7B_qLora_Finetuning.ipynb
I learn a lot from that, btw I'm not quite understand what I miss there

Issues

1. Generated instruction, the 1-3 characters get cut off. e.g . CRE get cut off and appear ATE instead of CREATE
2. Most of the time Generated instruction didn't match Ground truth.

Here's what I got.

The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.
Setting `pad_token_id` to `eos_token_id`:2 for open-end generation.
Prompt:
<s>
Generate a SQL query to create a table containing Movie information (title, release year, genre).
Not applicable
[INST]




Generated instruction:
ATE TABLE Movies (
    id INT PRIMARY KEY,
    title VARCHAR(255),
    release_year INT,
    genre VARCHAR(255)
);  [/INST] \n CREATE TABLE Movies (
    id INT PRIMARY KEY,
    title VARCHAR(255),
    release_year INT,
    genre VARCHAR(255)
);  [/INST]

Ground truth:
CREATE TABLE Movies (
 title VARCHAR(50) NOT NULL,
 release_year INT NOT NULL,
 genre VARCHAR(20)
);

Is this normal? Any hint to improve this?

Thanks

Optimize Fine-Tuning Hyperparameters

Description: Experiment with different hyperparameters and training strategies to optimize the fine-tuning process. Document your findings and suggest the best configuration for fine-tuning the model.

Skills Needed: Machine learning, Python, Git

Implement Response Context Awareness

Description: Work on improving the model's responses by making it more context-aware. Enhance the response generation to consider the previous parts of the conversation for more natural and coherent replies.

Skills Needed: Natural language processing, Python, Git

Compare Between Avalon-Llama-7b and Avalo-Mitsral-7b

Description

Conduct a thorough comparison between the Avalon-Llama-7b model and the Avalo-Mitsral-7b model. Evaluate their performance, capabilities, and responses to assess the strengths and weaknesses of each model.

Tasks

• Perform a detailed comparison of the Avalon-Llama-7b and Avalo-Mitsral-7b models.
• Assess their performance in various conversational scenarios.
• Evaluate their ability to provide meaningful responses and emulate sentience.
• Document the findings and insights from the comparison.
• Update the README to include a section summarizing the comparison results.