gmh5225/web-search-manticoresearch

Easy to use open source fast database for search | Good alternative to Elasticsearch now | Drop-in replacement for E in the ELK soon

Website • Downloads • Docs • Blog • Courses • Forum • Slack • Telegram (En) • Telegram (Ru) • Twitter

 

Introduction

❗Read recent blog post about Manticore vs Elasticsearch❗

Manticore Search is an easy to use open source fast database for search. Good alternative for Elasticsearch. What distinguishes it from other solutions is:

• It's very fast and therefore more cost-efficient than alternatives, for example Manticore is:
  • 182x faster than MySQL for small data (reproducible❗)
  • 29x faster than Elasticsearch for log analytics (reproducible❗)
  • 15x faster than Elasticsearch for small dataset (reproducible❗)
  • 5x faster than Elasticsearch for medium-size data (reproducible❗)
  • 4x faster than Elasticsearch for big data (reproducible❗)
  • up to 2x faster max throughput than Elasticsearch's for data ingestion on a single server (reproducible❗)
• Modern MPP architecture and smart query parallelization capabilities allow to fully utilize all your CPU cores to lower response time as much as possible, when needed.
• Powerful and fast full-text search which works fine for small and big datasets
• Traditional row-wise storage for small, medium and big size datasets
• Columnar storage support via the Manticore Columnar Library for bigger datasets (much bigger than can fit in RAM)
• Easy to use secondary indexes (you don't need to create them manually)
• Cost-based optimizer for search queries
• SQL-first: Manticore's native syntax is SQL. It speaks SQL over HTTP and uses the MySQL protocol (you can use your preferred MySQL client)
• Clients for PHP, Python, JavaScript, Java, Elixir and Go
• JSON over HTTP: to provide a more programmatic way to manage your data and schemas, Manticore provides a HTTP JSON protocol
• Written fully in C++: starts fast, doesn't take much RAM, and low-level optimizations provide good performance
• Real-time inserts: after an INSERT is made, the document is accessible immediately
• Interactive courses for easier learning
• Built-in replication and load balancing
• Can sync from MySQL/PostgreSQL/ODBC/xml/csv out of the box
• Not fully ACID-compliant, but supports transactions and binlog for safe writes

Craigslist, Socialgist, PubChem, Rozetka and many others use Manticore for efficient searching and stream filtering.

Manticore Search was forked from Sphinx 2.3.2 in 2017.

More features

• Full-text search and relevance:
  • Over 20 full-text operators and over 20 ranking factors
  • Custom ranking
• Other search capabilities:
  • Rich filtering functionality
  • Faceted search
  • Geo-spatial search
  • Spelling correction
  • Autocomplete
  • A wide range of functions for filtering and data manipulation
• Natural language processing (NLP):
  • Stemming
  • Lemmatization
  • Stopwords
  • Synonyms
  • Wordforms
  • Advanced tokenization at character and word level
  • Proper Chinese segmentation
  • Text highlighting
• Stream filtering using a "percolate" index
• High-availability:
  • Data can be distributed across servers and data-centers
  • Synchronous replication
  • Built-in load balancing
• Security:
  • https support
  • read-only mode
• Data storages:
  • row-wise - requires more RAM, provides faster performance
  • columnar - requires less RAM, still provides decent performance, but lower than the row-wise storage for some kinds of queries
  • docstore - doesn't require RAM at all, but allows only fetching original value, not sorting/grouping/filtering
• Optimizations:
  • Secondary indexes
  • Cost-based optimizer determines the most efficient execution plan of a search query
• Data types:
  • full-text field - inverted index
  • int, bigint and float numeric fields in row-wise and columnar fashion
  • multi-value attributes (array)
  • string and JSON
  • on-disk "stored" for key-value purpose
• Integrations:
  • Sync from MySQL and PostgreSQL
  • Sync from XML
  • Sync from CSV
  • With MySQL as a storage engine
  • With MySQL via FEDERATED engine
  • ProxySQL
  • Manticore Columnar Library

Installation

Docker

Docker image is available on Docker Hub.

To experiment with Manticore Search in Docker just run:

docker run --name manticore --rm -d manticoresearch/manticore && sleep 3 && docker exec -it manticore mysql && docker stop manticore

You can then: create an index, add data and run searches. For example:

create table movies(title text, year int) morphology='stem_en' html_strip='1' stopwords='en';

insert into movies(title, year) values ('The Seven Samurai', 1954), ('Bonnie and Clyde', 1954), ('Reservoir Dogs', 1992), ('Airplane!', 1980), ('Raging Bull', 1980), ('Groundhog Day', 1993), ('<a href="http://google.com/">Jurassic Park</a>', 1993), ('Ferris Bueller\'s Day Off', 1986);

select highlight(), year from movies where match('the dog');

select highlight(), year from movies where match('days') facet year;

select * from movies where match('google');

When you exit from the MySQL client, it stops and removes the container, so use this way only for testing / sandboxing purposes.

Read the full instruction for the docker image for more details including our recommendations on running it in production.

Packages

Ubuntu, Debian, Centos, Windows and MacOS packages are here.

YUM repo for RHEL/Centos/Amazon Linux

sudo yum install https://repo.manticoresearch.com/manticore-repo.noarch.rpm
sudo yum install manticore manticore-columnar-lib

APT repo for Ubuntu/Debian/Mint

wget https://repo.manticoresearch.com/manticore-repo.noarch.deb
sudo dpkg -i manticore-repo.noarch.deb
sudo apt update
sudo apt install manticore manticore-columnar-lib

Homebrew on MacOS

brew install manticoresearch

Windows

See instruction here.

MacOS tarball

See instruction here.

Documentation and community sites

• Documentation
• Interactive courses
• Manticore Community Forum
• Public Slack chat
• Public Telegram chat (En)
• Public Telegram chat (Ru)
• Bug tracker

Third-party integrations

• https://dev.tiki.org/Manticore-Search - TikiWiki CMS Groupware integration
• https://github.com/EvilFreelancer/laravel-manticoresearch - Laravel ManticoreSearch plugin
• https://www.mediawiki.org/wiki/Extension:SphinxSearch - extension for MediaWiki
• https://github.com/gordonbanderson/silverstripe-manticore-search - integration for SilverStripe
• https://drwho.virtadpt.net/archive/2022-05-13/combining-manticore-and-searx/ - integration for SearX
• https://github.com/dimv36/sphinxlink - Extension for PostgreSQL for running search queries and returning data as PostgreSQL tables

How we can support you

Should your company require any help - we provide full-cycle services in the areas of Sphinx and Manticore Search:

• Audit
• Support
• Consulting
• Development
• Training

More details here

❤️ How you can support Manticore Search

Manticore Search is a GPLv2-licensed Open Source project with development made possible by support from our core team, contributors, and sponsors. Building premium Open Source software is not easy. If you would like to make sure Manticore Search stays free, here is how you can help the project:

• Donation through PayPal
• Become our client and let us help you