thedatatribune/dypixa

Turning words into lively shades!

dyPixa

Dynamically generating abstract images using Computer Vision, Machine Learning, and Sentiment Analysis

dyPixa, aka Dynamic Pixels, is an open-source project that aims to develop a tool combining the power of computer vision, machine learning, and natural language processing to create abstract images based on text input and sentiment analysis. With dyPixa, you can generate stunning visuals by harnessing the emotions expressed in the text.

Table of Contents

• Introduction
• Features
• Getting Started
• Usage
• Contributing
• License

Introduction

dyPixa is a project that enables you to analyze Multilingual text, perform sentiment analysis, and use the emotions expressed in the text to generate abstract images with carefully selected color combinations. It would leverage state-of-the-art machine learning models and image processing techniques to achieve this. Additionally, dyPixa allows you to overlay the input text onto the generated abstract image, creating visually striking compositions.

Features

• Multilingual Text and Sentiment Analysis: dyPixa can analyze Hindi text and determine its sentiment, whether it's positive, negative, or neutral.

• Color Combination Model: Train a machine learning model using a dataset of images paired with text descriptions to learn the most suitable color combinations for different sentiments.

• Abstract Image Generation: Generate abstract images based on the input text's sentiment, utilizing the color combinations learned by the model.

• Text Overlay: Overlay the input text onto the generated abstract image, allowing you to create visually appealing compositions that convey the text's emotion.

Getting Started

To get started with dyPixa, follow these steps:

1. Fork this repository and clone using following command:

$ git clone https://github.com/<your-username>/dyPixa.git  
$ cd dyPixa  

2. Install Dependencies:

• Install the required Python libraries by running (the requirements.txt is updated with growth of the project):

  $ pip install -r requirements.txt

• Download Pre-trained Models (applicable in future as the project grows):
  Depending on the project's requirements, you might need to download pre-trained models for sentiment analysis or color combination generation. Check the project documentation for instructions on acquiring these models.

3. Run/Enhance the Project:

• Follow the project-specific instructions in the documentation (to be updated) to use dyPixa for text analysis, color combination generation, abstract image creation, and text overlay. Once you are inthe working directory, i.e., dyPixa; you can also start enhancing the code. Being open-source, maintainers would love to merge your contributions.

Usage

Being in an early phase of the development, the usage guide for dyPixa is yet to be populated. However, with this tool, one would be able to do:

1. Text Analysis and Sentiment Analysis

2. Color Combination Generation

3. Abstract Image Generation

4. Text Overlay

Contributing

Contributions to dyPixa are welcome! Whether you want to improve existing features, add new functionality, or report issues, please follow the contribution guidelines outlined in the project's CONTRIBUTING.md file.

License

This project is licensed under the MIT License. Feel free to use, modify, and distribute dyPixa according to the terms of the license.

---

Note for contributors: This README.md is supposed to be updated as per any new feature/changes introduced. Provide clear and comprehensive instructions to help users get be familiar.

dypixa's Issues

Dataset: Collection and Description

Dataset Requirements 📦📋

TL; DR 🥱

This issue is one of the great starting point for the beginners in opensource community, here you can:

• share the authentic data
• contribute with new APIs for collecting the same
• even provide (original) scripts (in any preferred language) for data collection and/or preparation like cleaning

---

Issue Description:

In the context of the dyPixa project, this task revolves around the crucial need to gather and comprehensively document datasets for training and testing the machine learning models. This issue addresses the following key aspects:

1. Data Collection Scripts: Define a systematic approach and python (preferred) code for sourcing diverse datasets. This may include acquiring text data from social media, product reviews, and news articles, and images with associated sentiments from public image repositories.
2. Dataset Documentation: Document (in #7 once uploaded) or raise the concerns related to (existing or required) dataset specifics, source, size, language distribution, and preprocessing. Refer to Issue #7 for detailed documentation guidelines.
3. Data Quality Assurance: Ensure dataset integrity and consistency and is taken from authentic sources.
4. Multilingual Considerations: Explore strategies for multilingual datasets.
5. Collaboration with Contributors: Engage contributors in dataset sourcing and curation.

Types of Data Needed:

For the NLP and color suggestion models to be highly usable and effective, the following types of data should be considered:

• Text Data:

  • Social media posts
  • Product reviews
  • News articles
  • Sentiment-labeled text in English and Hindi
  • Multilingual text data to enhance language support

• Image Data:

  • Images with associated sentiment labels
  • Diverse images representing a wide range of emotions
  • Abstract images showcasing various color combinations

---

By addressing these components and collecting the appropriate types of data, this issue will lay the foundation for robust machine learning model development and further enhancements in the dyPixa project. Your contributions here will greatly advance the project's capabilities. 🚀🌈

Documentation

Issue Title: Documentation

Description:
📖 Overview
It's crucial to ensure that the documentation of dyPixa remains comprehensive, up-to-date, and user-friendly. Documentation serves as the gateway for contributors and users to understand and utilize the full potential of dyPixa. This issue is dedicated to improving our project's documentation across various aspects.

📋 Tasks

• User Guide Enhancement: Expand and refine the user guide to provide clear, step-by-step instructions for using dyPixa. Include examples and best practices for generating abstract images, utilizing the color harmony model, and best practices to provide the inputs.

• Contributor Guide Update: Ensure our contributor guide is concise and offers a smooth onboarding experience for new contributors. Explain our development workflow, coding standards, and how to submit contributions effectively.

• API Documentation: We aim to develop APIs for each of the four main sub-parts/features of dyPixa as we progress. Simultaneously, we will document the project's API, detailing functions and integration endpoints, along with usage examples for developers implementing dyPixa in their applications.

• Installation Instructions: Improve the installation guide to cover different platforms and deployment scenarios. Ensure it is beginner-friendly and includes troubleshooting tips.

• FAQ Section: Create a Frequently Asked Questions (FAQ) section to address common queries and issues users and contributors may encounter.

🛠️ Expected Outcome
By addressing this documentation issue, we aim to provide a more accessible and informative resource for the community. This will empower both users and contributors to make the most of dyPixa's capabilities, fostering a stronger and more vibrant project ecosystem.

📌 Note
If you choose to work on this issue, please fork the repository, create a new branch for your changes, and submit a pull request once your updates are ready. Your contributions to improving our documentation are highly valued and appreciated! 🙌

Let's make dyPixa even more accessible and user-friendly together! 🚀🌟

[Maintenance] Update `requirements.txt`

Description 🛠️

It's time to ensure our project's dependencies are up-to-date. Let's update the requirements.txt file to keep our codebase fresh and reliable. Your contribution to this essential maintenance task is greatly appreciated! 🚀

---

Note: Do ensure that you take into account the requirements of all available Python code, which can be found in the code/ directory across various subdirectories.

[CI/CD] Containerize the Color Recommendation 🐳

Description: 📝

This issue aims to create a Docker container for the solution provided in #30 within the dyPixa project. It addresses the requirement of developing a user-friendly front-end interface for users to input text, transmitting this input to the backend/API, and receiving color shade recommendations in hexadecimal form (#FFFFFF for white) as the response.

What to do?

1. 🐳 Create Docker Container:

   • Develop a Docker container that must import the solutions of #30 and #40 .
   • The front end needs to be updated to take image input as well (if not already there).

2. 📦 Dependency Management:

   • Include all required dependencies within the container.

3. 🔄 CI/CD Integration:

   • Integrate container creation into the CI/CD pipeline.
   • Provide corresponding Dockerfile and docker-compose.yml.

4. 🧪 Testing and Documentation:

   • Implement testing and provide clear usage documentation.

User Interface

🌐 dyPixa UI Enhancement 🖌️

Developing an open-source web app, dyPixa, powered by ML and LLMs. APIs enable color scheme recommendations and dynamic image generation (Issue #4).

Requirements

We aim to address various UI needs:

1. Text-Based Color Recommendation 🌈

   • Create a web portal for text input, fetching color suggestions via API (JSON).
   • Display results intuitively on the same portal.

2. Text-to-Image Conversion 📝🖼️

   • Input text to trigger ML/CV model for image generation.
   • Render and allow users to download/share images.

3. Color-to-Image Generation 🎨🖼️

   • Enable color selection and input for image generation (#4).
   • Output an image based on color input.
   • Optionally, accept an image input for enhanced image generation.

[Fix] Update code for ML model

The following changes are requested on #59 to integrate the solution with UI:

1. Move the notebook (.ipynb) from the models to the code directory.
2. Using the provided jupyter notebook, save the final model (you may use pickle) and place it in models directory.

---

Convert both train and test data in a uniform format (either in Excel or CVS only).

[NLP] Create an NLP Model

Description: 📝

We need an NLP model to process English short poems and extract sentiments, emphasizing metaphor recognition for accurate sentiment analysis.

Requirement 🚀

1. NLP model for sentiment extraction from English short poems.
2. Refer to issues #6 and #26 for dataset details. Contributions to meet dataset requirements are appreciated. 📊📚

[Fix] Improve Code Error Handling

Issue Description

This issue involves enhancing error handling in the dyPixa project's code (primarily Python). Effective error handling is essential for application stability and user experience.

Key Focus Areas

1. Review and improve existing error handling.
2. Enhance error messages for clarity.
3. Implement robust exception handling.
4. Ensure comprehensive error logging.
5. Thoroughly test error-handling scenarios.

Contributors are invited to improve dyPixa's error management, enhancing application reliability. Your contributions will elevate the project's quality. 🚀🛠️

[Dataset] Contribute Emotion Dataset for Color Generation 🌈

Description 🌈🎨

Our Quest: Let's supercharge dyPixa's creative potential! We're on a mission to build a vibrant dataset that connects emotions to colors, and we need your help to make it happen. 🚀

What's Needed? 📸🎨

1. Emotional Imagery: We're looking for images or thumbnails that express each of the 28 unique emotions. These visuals will breathe life into our dataset.

2. Color Splash: For each emotion, we want to collect a colorful array of at least 5 hues. These will be the building blocks of our dynamic color palettes.

Your Creative Input! 🎨💡

1. Shades of Emotion: Remember, there's a world of color sets that can convey the same emotion, so feel free to explore the possibilities.

2. Organized Magic: Treat each color set that aligns with a specific emotion as a distinct data entry.

3. Effortless Extraction: To make your contribution smoother, you can use this handy script by @thevectorguy to automatically fetch colors from your provided images or thumbnails.

Let's Paint the World with Emotions!

By contributing to this dataset, you're not only adding color to dyPixa's palette but also unleashing a world of creative emotions. Join us in this colorful journey and be a part of the dyPixa magic! 🎨✨

Ready to dive in? Contribute to the dyPixa Emotion Dataset and let's make emotions burst with life through colors! 🚀

---

Note: Questions or need guidance? You can join dyPixa's lively community at Discord for further discussions and queries. Your contributions mean the world to us! 🙌

[Dataset] Poetry dataset is required

Description: 📝

This project requires an NLP model trained on a poetry dataset, encompassing different languages with a current focus on English and Hindi. The dataset should meet the following constraints:

1. English short poems.
2. Hindi short poems.
3. The datasets must be of high quality, sufficiently large, and diverse.
4. Ensure that these short poems contain figures of speech.

For longer poems, consider the following contributions:

1. Provide scripts to preprocess the data and divide lengthy poems into shorter segments.
2. Provide the processed dataset.

---

Note:

• It's crucial to collect short poems as the ultimate goal of dyPixa is to render the input text on the generated image.
• Utilize publicly available datasets whenever possible.
• If providing poems not openly available for analysis, ensure proper credit is given to the authors by adding an additional field for author names.
• Poems may be the intellectual property of their authors; therefore, obtain authors' consent before uploading data to this repository.
• For further dataset requirements, refer to #6 for additional details.

[Dataset] Crafting the Colors of Emotions 🌈🎨

Description:

Our mission is clear: we need to build a comprehensive dataset that harmonizes dominant colors with various emotions and feelings. To achieve this, we've outlined a set of reference steps:

1. Image Quest: Let's embark on a quest to fetch and collect images or thumbnails tagged with specific sentiments and emotions.
2. Python Magic: We'll utilize the power of Python (our preferred choice) to load these images and extract the most dominant colors from each artwork.
3. Color Script: We have a nifty tool in our arsenal. You can make use of the ColorExtraction.py script available within dyPixa's resources to list the vibrant colors.
4. JSON Palette: Store the beautiful color sets for a unique image or emotion in JSON format. Embrace the creativity, as the number of shades can vary from one image to another.

The colors of each image could be stored keeping the MD5 hash of the image as its key. The JSON data can the following structure:

{
    "emotions": {
            "ImageHash": {
                        "HEX": [
                                "shade1",
                                "shade2",
                                "....",
                                "shadeN"
                                ],
                        "RGB": {
                            "shade1": ["R", "G", "B"],
                            "shade2": ["R", "G", "B"],
                            "...": ["R", "G", "B"],
                            "shadeN": ["R", "G", "B"]
                        }
            }
    }
}

This dataset holds immense potential. We'll harness it to train machine learning models, enabling us to recommend the perfect palette of colors for any given sentiment. 🎨🚀

Let's paint the world with emotions! Your contributions are a stroke of genius. 🙌

ML Model

dyPixa and ML Models 🤖

This project relies heavily on creating, deploying, and utilizing various machine learning models, primarily for:

1. NLP Analysis 📝
2. Color Recommendations (supervised learning) 🌈
3. Clustering 🔍

Why Machine Learning? 🤔

1. Text Sentiment Analysis with NLP: Crucial for extracting sentiments from diverse text in multiple languages (English and Hindi). The model should understand languages and generalize sentiments effectively. 🌐🗣️
2. Sentiments-to-Color Recommendation: Colors significantly impact emotions (research, blog). Using supervised learning on sentiment-related images, the ML model will suggest optimal color combinations for enhancing content engagement, with broader applications. 🎨🖼️
3. Clustering: To diversify color recommendations, preventing repetitive solutions. ML models can create color clusters suggested by previous models to enhance color selection. 🔄🌈

Accepted Contributions... 🤝✅

This issue focuses on developing new ML and NLP models and associated code for:

• Language recognition and text processing 🗣️📝
• Multilingual sentiment analysis 🌐📊
• Mapping sentiments to multiple hex color codes 🌈🔢
• Generating diverse color combinations 🌟🌈

Contributors are encouraged to tackle related sub-problems while addressing issue #4. 💡🔧

---

Note: 📌 The necessary information related to the data to build/test these models could be found in issue #6. It is appreciated to contribute with data, as well.

[UI/Frontend] Text Sentiment Extraction

A UI is required to integrate the emotion classification model based on go-emotions, which is provided in the ml_nlp directory.

The UI should include the following components:

1. An input field for text data.
2. A button to trigger the findEmotions() function.
3. Proper error handling for empty input.
4. Display the retrieved JSON data in the most appropriate fashion.

[UI/Frontend] Color Recommendation Portal

Text-Based Color Recommendation 🌈

We require a user-friendly front-end interface 🖥️ to facilitate the user with a form for inputting text. This text input will be transmitted to the backend/API, and the received response should consist of a set of color shades 🎨 (in hexadecimal form, e.g., #FFFFFF for white).

Key Tasks:

• Develop a web portal for text input, enabling the retrieval of color suggestions.
• Present the results in a user-friendly and intuitive manner on the same portal.
• We encourage contributions based on popular Python frameworks such as Django or Flask 🐍.
• Maintain clean and comprehensible code, including helpful comments when necessary.

About the Processing:

• Incorporate dummy "hello world" scripts into the backend processing, which will later be substituted with original function calls based on the developed ML/NLP models 🤖.
• Expect the initiation of an API call to transmit the input text and retrieve results, preferably in JSON format 📤.

Your contributions to this feature will enhance the user experience by providing dynamic color recommendations, enriching the functionality of the dyPixa project. 🚀🌈

---

NOTE: Refer to Issue #5 for more details.

[Graphics] Create a Captivating Logo for dyPixa

Calling All Design Enthusiasts! 🎨🖌️

dyPixa, our innovative project at the intersection of computer vision, machine learning, and natural language processing, is looking for a unique and captivating logo to represent its identity. We believe that a visually appealing logo can truly capture the essence of our project and make a lasting impression.

🎯 What We're Looking For
We're open to your creative genius! Whether it's abstract art, a clever combination of letters, or a symbol that represents our project's mission, we're excited to see your ideas. Here are some key guidelines:

• Innovative: We want a logo that feels cutting-edge and forward-thinking.
• Versatile: It should look great in both color and grayscale.
• Memorable: Something that sticks in the mind and makes people think of dyPixa.

💡 How to Participate

1. Create your logo design.
2. Submit your design through a sub-directory of logo/.
3. Feel free to share your design process or inspiration!

🚀 Let's Get Creative
Together, we can make dyPixa visually unforgettable. We're excited to see your designs and grateful for your contributions to our project's identity. 🌌

Thank you for being part of the dyPixa community! 🙌

---

Note: 📌 If you have any questions or need further guidance on this issue, join us at Discord.

[Dataset] Color Extraction using Python

Description: 📝

We need a simple Python program to extract dominant colors from an input image (provided as a file path) and present them in hex format (e.g., white = #FFFFFF).

Key Focus Areas: 🗝️🛠️

1. The program should support various image file formats (e.g., jpg, png, bmp).
2. Instead of extracting all colors, it should identify and report the top colors based on their frequency in the image.
3. In the code, please include clear and meaningful comments to enhance comprehension.

---

Note: 📌 Solving this issue will contribute to the advancement of #6.

Text Overlay on Images 🎨📚

Description: 📜

dyPixa's project goals is to enable the dynamic overlay of text onto generated images, fostering the creation of emotionally expressive compositions. This task is a crucial element within our broader project.

As you work on this issue, please consider the following constraints to ensure the best results:

1. Full Coverage: The entire input text should overlay the image, ensuring that no part is cut off. 📌
2. Visibility: The overlaid text must be clearly visible, maintaining readability and legibility. 👁️
3. Dynamic Dimensions: Keep in mind that the images can come in one of three different ratios: 1:1, 3:4, and 9:16. Your solution should gracefully adapt to these varying dimensions. 📐
4. Text Font Size: While there are no specific constraints regarding text font size, it should be chosen to enhance the overall visual appeal while adhering to visibility and readability guidelines. ✒️

Emotionally Expressive Compositions 🎨

This issue is a key step in bringing emotion and creativity to life through text and images. Your contributions are greatly appreciated as we work together to make dyPixa truly extraordinary. 🚀🌈