pahadon/search-emf-talks

Semantic Search over EMF Talk Transcripts

Semantic Search

Allow natural language search on user content like notes, images, transactions using transformer ML models

User can interface with semantic-search via the API or Emacs. All search is done locally*

Demo

demo.mp4

Setup

1. Clone

git clone https://github.com/debanjum/semantic-search && cd semantic-search

2. Configure

• [Required] Update docker-compose.yml to mount your images, org-mode notes and beancount directories
• [Optional] Edit application configuration in sample_config.yml

3. Run

docker-compose up -d

Note: The first run will take time. Let it run, it’s mostly not hung, just generating embeddings

Use

• Semantic Search via API
  • See Semantic Search API Docs
  • Query
  • Regenerate Embeddings
  • Configure Application
• Semantic Search via Emacs
  • Install semantic-search.el
  • Run M-x semantic-search <user-query>

Upgrade

docker-compose build

Troubleshooting

• Symptom: Errors out with “Killed” in error message
  • Fix: Increase RAM available to Docker Containers in Docker Settings
  • Refer: StackOverflow Solution, Configure Resources on Docker for Mac
• Symptom: Errors out complaining about Tensors mismatch, null etc
  • Mitigation: Delete content-type > image section from docker_sample_config.yml

Miscellaneous

• The experimental chat API endpoint uses the OpenAI API
  • It is disabled by default
  • To use it add your openai-api-key to config.yml

Development Setup

Setup on Local Machine

1. Install Dependencies

1. Install Python3 [Required]
2. Install Conda [Required]
3. Install Exiftool [Optional]

   sudo apt-get -y install libimage-exiftool-perl
       

2. Install Semantic Search

git clone https://github.com/debanjum/semantic-search && cd semantic-search
conda env create -f config/environment.yml
conda activate semantic-search

3. Configure

• Configure files/directories to search in content-type section of sample_config.yml
• To run application on test data, update file paths containing /data/ to tests/data/ in sample_config.yml
  • Example replace /data/notes/*.org with tests/data/notes/*.org

4. Run

Load ML model, generate embeddings and expose API to query notes, images, transactions etc specified in config YAML

python3 -m src.main -c=config/sample_config.yml -vv

Upgrade On Local Machine

cd semantic-search
git pull origin master
conda deactivate semantic-search
conda env update -f config/environment.yml
conda activate semantic-search

Acknowledgments

• MiniLM Model for Asymmetric Text Search. See SBert Documentation
• OpenAI CLIP Model for Image Search. See SBert Documentation
• Charles Cave for OrgNode Parser
• Sven Marnach for PyExifTool