Giter Site home page Giter Site logo

danielefedeli / lyra Goto Github PK

View Code? Open in Web Editor NEW

This project forked from askorama/orama

0.0 0.0 0.0 13.45 MB

๐ŸŒŒ Fast, in-memory, typo-tolerant, full-text search engine written in TypeScript. Now in beta.

Home Page: https://nearform.github.io/lyra

License: Other

Shell 0.04% TypeScript 95.54% Dockerfile 1.76% Rust 2.66%

lyra's Introduction

Lyra, the edge search experience

Tests

Join Lyra's Slack channel

If you need more info, help, or want to provide general feedback on Lyra, join the Lyra Slack channel

Installation

You can install Lyra using npm, yarn, pnpm:

npm i @lyrasearch/lyra
yarn add @lyrasearch/lyra
pnpm add @lyrasearch/lyra

Or import it directly in a browser module:

<html>
  <body>
    <script type="module">
      import { create, search, insert } from "https://unpkg.com/@lyrasearch/lyra@latest/dist/esm/src/lyra.js";

      // ...
    </script>
  </body>
</html>

See builds for details about the various builds packaged with Lyra.

Read the complete documentation at https://docs.lyrasearch.io/.

Usage

Lyra is quite simple to use. The first thing to do is to create a new database instance and set an indexing schema:

import { create, insert, remove, search } from "@lyrasearch/lyra";

const db = await create({
  schema: {
    author: "string",
    quote: "string",
  },
});

Lyra will only index string properties, but will allow you to set and store additional data if needed.

Once the db instance is created, you can start adding some documents:

await insert(db, {
  quote:
    "It is during our darkest moments that we must focus to see the light.",
  author: "Aristotle",
});

await insert(db, {
  quote:
    "If you really look closely, most overnight successes took a long time.",
  author: "Steve Jobs",
});

await insert(db, {
  quote:
    "If you are not willing to risk the usual, you will have to settle for the ordinary.",
  author: "Jim Rohn",
});

await insert(db, {
  quote: "You miss 100% of the shots you don't take",
  author: "Wayne Gretzky - Michael Scott",
});

If you have a large number of documents, we highly recommend using the insertBatch function, which prevents the event loop from blocking. This operation is asynchronous and returns a promise:

await insertBatch(db, [
  {
    quote:
      "It is during our darkest moments that we must focus to see the light.",
    author: "Aristotle",
  },
  {
    quote:
      "If you really look closely, most overnight successes took a long time.",
    author: "Steve Jobs",
  },
  {
    quote:
      "If you are not willing to risk the usual, you will have to settle for the ordinary.",
    author: "Jim Rohn",
  },
  {
    quote: "You miss 100% of the shots you don't take",
    author: "Wayne Gretzky - Michael Scott",
  },
]);

After the data has been inserted, you can finally start to query the database.

const searchResult = await search(db, {
  term: "if",
  properties: "*",
});

In the case above, you will be searching for all the documents containing the word if, looking up in every schema property (AKA index):

{
  elapsed: 184541n, // Elapsed time in nanoseconds
  hits: [
    {
      id: '41013877-56',
      score: 0.025085832971998432,
      document: {
        quote: 'If you really look closely, most overnight successes took a long time.',
        author: 'Steve Jobs'
      }
    },
    {
      id: '41013877-107',
      score: 0.02315615351261394,
      document: {
        quote: 'If you are not willing to risk the usual, you will have to settle for the ordinary.',
        author: 'Jim Rohn'
      }
    }
  ],
  count: 2
}

You can also restrict the lookup to a specific property:

const searchResult = await search(db, {
  term: "Michael",
  properties: ["author"],
});

Result:

{
  elapsed: 172166n,
  hits: [
    {
      id: '41045799-144',
      score: 0.12041199826559248,
      document: {
        quote: "You miss 100% of the shots you don't take",
        author: 'Wayne Gretzky - Michael Scott'
      }
    }
  ],
  count: 1
}

If needed, you can also delete a given document by using the remove method:

await remove(db, "41045799-144");

Lyra exposes a built-in formatNanoseconds function to format the elapsed time in a human-readable format:

import { formatNanoseconds } from "@lyrasearch/lyra";

const searchResult = await search(db, {
  term: "if",
  properties: "*",
});

console.log(`Search took ${formatNanoseconds(searchResult.elapsed)}`);
// Search took 164ฮผs

Language

Lyra supports multiple languages. By default, it will use the english language,

You can specify a different language by using the defaultLanguage property during Lyra initialization.

By default, Lyra will analyze your input using an English Porter Stemmer function.
You can replace the default stemmer with a custom one, or a pre-built one shipped with the default Lyra installation.

Example using ESM (see builds below):

import { create } from "@lyrasearch/lyra";
import { stemmer } from "@lyrasearch/lyra/dist/esm/stemmer/it";

const db = await create({
  schema: {
    author: "string",
    quote: "string",
  },
  defaultLanguage: "italian",
  components: {
    tokenizer: {
      stemmingFn: stemmer,
    },
  },
});

Example using CJS (see builds below):

const { create } = require("@lyrasearch/lyra");
const { stemmer } = require("@lyrasearch/lyra/dist/esm/stemmer/it");

const db = await create({
  schema: {
    author: "string",
    quote: "string",
  },
  defaultLanguage: "italian",
  components: {
    tokenizer: {
      stemmingFn: stemmer,
    },
  },
});

Right now, Lyra supports 23 languages and stemmers out of the box:

  • Armenian
  • Arabic
  • Danish
  • Spanish
  • English
  • Finnish
  • French
  • German
  • Greek
  • Hindi
  • Hungarian
  • Indonesian
  • Italian
  • Irish
  • Dutch
  • Nepali
  • Norwegian
  • Portuguese
  • Romanian
  • Russian
  • Serbian
  • Swedish
  • Turkish

Builds

Lyra is packaged with ES modules, CommonJS, and generic browser builds.

In most cases, simply import or require @lyrasearch/lyra and your environment will choose the most appropriate build โœจ. In some circumstances, you may need to import or require certain files (such as stemmers). The following builds are included in the Lyra package:

path build
dist/esm ESNext build using ES modules. Use this for most modern applications (node.js, vite.js, browser modules, etc.)
dist/cjs ESNext build using CommonJS (require). Use this for environments that don't support ES modules.
dist/browser ES2019 build using CommonJS (require). Use this for environment that don't support modern ESNext language constructs, such as webpack 4 (used by Expo). Note, this build will be chosen by default in webpack environments such as Next.js.

Hooks

When dealing with asynchronous operations, hooks are an excellent mechanism to intercept and perform operations during the workflow. Lyra supports hooks natively. The create function allows you to specify a sequence of hooks.

import { create } from "@lyrasearch/lyra";

const db = await create({
  schema: {},
  hooks: {
    // HERE
  },
});

Important: The hooks run in the same context as the main function execution. It means, that if your hook takes X milliseconds to resolve, the Lyra function will take X + Y (where Y = Lyra operation).

afterInsert hook

The afterInsert hook is called after the insertion of a document into the database. The hook will be called with the id of the inserted document.

Example:

import { create, insertWithHooks } from "@lyrasearch/lyra";

async function hook1 (id: string): Promise<void> {
  // called before hook2
}

function hook2 (id: string): void {
  // ...
}

const db = await create({
  schema: {
    author: "string",
    quote: "string",
  },
  hooks: {
    afterInsert: [hook1, hook2],
  },
});

await insertWithHooks(db, { author: "test", quote: "test" })

License

Lyra is licensed under the Apache 2.0 license.

lyra's People

Contributors

aldotele avatar bozzelliandrea avatar castarco avatar codyzu avatar danielefedeli avatar frenzarectah avatar giovannioggioni avatar ilteoood avatar ishibi avatar jkomyno avatar lbrdan avatar marco-ippolito avatar mateonunez avatar matteogheza avatar matteoscaramuzza avatar micheleriva avatar mrbrianevans avatar optic-release-automation[bot] avatar paolo-cargnin avatar rafaelgss avatar shogunpanda avatar sidwebworks avatar simoneb avatar simonireilly avatar thomscoder avatar tianjos avatar victortosts avatar yyl2020 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.