Giter Site home page Giter Site logo

bkbase-plugin / quickwit Goto Github PK

View Code? Open in Web Editor NEW

This project forked from quickwit-oss/quickwit

0.0 0.0 0.0 8.67 MB

Cloud-native search engine for log management & analytics

Home Page: https://quickwit.io

License: Other

Shell 0.77% JavaScript 0.73% Python 0.06% Rust 93.32% TypeScript 4.38% CSS 0.04% Makefile 0.15% HTML 0.08% PLpgSQL 0.16% Dockerfile 0.33%

quickwit's Introduction

CI codecov Contributor Covenant License: AGPL V3 Twitter Follow Discord Rust



Quickwit Log Management & Analytics Quickwit Log Management & Analytics

Search more with less

The new way to manage your logs at any scale


โ—Disclaimer: you are reading the README of Quickwit 0.3 version that will be shipped in May 2022.

Quickwit is the next-gen search & analytics engine built for log management. It is a highly reliable & cost-efficient alternative to Elasticsearch.



๐Ÿ’ก Features

  • Index data persisted on object storage
  • Ingest JSON documents with or without a strict schema
  • Ingest & Aggregation API Elasticsearch compatible
  • Lightweight Embedded UI
  • Runs on a fraction of the resources: written in Rust, powered by the mighty tantivy
  • Works out of the box with sensible defaults
  • Optimized for multi-tenancy. Add and scale tenants with no overhead costs
  • Distributed search
  • Cloud-native: Kubernetes ready
  • Add and remove nodes in seconds
  • Decoupled compute & storage
  • Sleep like a log: all your indexed data is safely stored on object storage (AWS S3...)
  • Ingest your documents with exactly-once semantics
  • Kafka-native ingestion
  • Search stream API that notably unlocks full-text search in ClickHouse

๐Ÿ”ฎ Upcoming Features

  • Ingest your logs from your object storage
  • Distributed indexing
  • Support for tracing
  • Native support for OpenTelemetry

๐Ÿ”Ž Uses & Limitations

โœ… ย  When to use โŒ ย  When not to use
Your documents are immutable: application logs, system logs, access logs, user actions logs, audit trail (logs), etc. Your documents are mutable.
Your data has a time component. Quickwit includes optimizations and design choices specifically related to time. You need a low-latency search for e-commerce websites.
You want a full-text search in a multi-tenant environment. You provide a public-facing search with high QPS.
You want to index directly from Kafka. You want to re-score documents at query time.
You want to add full-text search to your ClickHouse cluster.
You ingest a tremendous amount of logs and don't want to pay huge bills.
You ingest a tremendous amount of data and you don't want to waste your precious time babysitting your cluster.

โšก Getting Started

Let's download and install Quickwit.

curl -L https://install.quickwit.io | sh

You can now move this executable directory wherever sensible for your environment and possibly add it to yourย PATHย environment. You can also install it viaย other means.

Take a look at our Quick Start to do amazing things, like Creating your first index or Adding some documents, or take a glance at our full Installation guide!

๐Ÿ“š Tutorials

๐Ÿ’ฌ Community

๐Ÿ™‹ FAQ

How is Quickwit different from traditional search engines like Elasticsearch or Solr?

The core difference and advantage of Quickwit is its architecture that is built from the ground up for cloud and log management. Optimized IO paths make search on object storage sub-second and thanks to the true decoupled compute and storage, search instances are stateless, it is possible to add or remove search nodes within seconds. Last but not least, we implemented a highly-reliable distributed search and exactly-once semantics during indexing so that all engineers can sleep at night. All this slashes costs for log management.

How does Quickwit compare to Elastic in terms of cost?

We estimate that Quickwit can be up to 10x cheaper on average than Elastic. To understand how, check out our blog post about searching the web on AWS S3.

What license does Quickwit use?

Quickwit is open-source under the GNU Affero General Public License Version 3 - AGPLv3. Fundamentally, this means that you are free to use Quickwit for your project, as long as you don't modify Quickwit. If you do, you have to make the modifications public. We also provide a commercial license for enterprises to provide support and a voice on our roadmap.

What is Quickwit's business model?

Our business model relies on our commercial license. There is no plan to become SaaS in the near future.

๐Ÿช„ Third-Party Integration

Ingest Logs with Kafka Ingest Logs with Kafka ย  Metastore Backed by Postgresql ย ย  Store logs on AWS S3 Store logs on AWS S3 ย  ย  Ingest logs with Amazon Kinesis ย  Integrate with Minio ย ย  Collect Logs on Kubernetes cluster Integration with Ceph Integration with Ceph

๐Ÿค Contribute and spread the word

We are always super happy to have contributions: code, documentation, issues, feedback, or even saying hello on discord! Here is how you can help us build the future of log management:

โœจ And to thank you for your contributions, claim your swag by emailing us at hello at quickwit.io.

๐Ÿ”— Reference

quickwit's People

Contributors

boraarslan avatar ddelemeny avatar dependabot[bot] avatar dzvon avatar evanxg852000 avatar fmassot avatar fulmicoton avatar guilload avatar iaziz786 avatar infiniteregrets avatar kezhenxu94 avatar kination avatar linxgnu avatar llogiq avatar maximkpa avatar mosuka avatar pseitz avatar r-zenine avatar saroh avatar xvello avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.