msamwelmollel/gliner_local

Run locally GLiNER

Enhancing Retrieval-Augmented Generation: Tackling Polysemy, Homonyms, and Entity Ambiguity with GLiNER for Improved Performance

Introduction

The advent of Large Language Models (LLMs) has significantly impacted the field of natural language processing, offering solutions to a wide array of general problems. However, these models often struggle with domain-specific tasks, emerging knowledge, and especially with issues like polysemy, homonyms, and entity ambiguity, leading to inaccurate or hallucinated responses. This project presents the enhanced Retrieval-Augmented Generation (RAG) systems using GLiNER, a state-of-the-art Named Entity Recognition (NER) tool, in combination with Llama-Index and Mistral, to address these challenges effectively.

Installation

Before you start, ensure your system meets the following requirements:

Python 3.8 or higher pip for Python 3

Usage

This project involves two significant steps to enhance the RAG system: entity recognition, query refinement, and leveraging LLMs for improved performance. Follow these steps to set up and use the system:

Step 1: Setup GLiNER Clone the GLiNER repository or install it via HuggingFace:

Step 2: Run Clone my repository and then run RAG_NER.py:

Contributing

We welcome contributions from the community. If you'd like to improve the project or suggest enhancements, please follow reach my email: [email protected]