typesense/typesense-js

JavaScript / TypeScript client for Typesense

typesense-js

Javascript client library for accessing the Typesense HTTP API.

This library can be used both on the server-side and on the client-side. The library's source is in ES6 and during build time, we transpile it to ES5 and generate two artifacts - one that can be used on the server-side and another that uses Browserify and can be used on the client side.

Installation

Option 1: Install via npm

npm install --save typesense

Install peer dependencies:

npm install --save @babel/runtime

Note: @babel/runtime is very a common dependency among many JS libraries. So instead of each library adding it as a dependency independently (which will cause multiple instances of @babel/runtime to be installed increasing bundle size), Babel's recommendation is that the libraries ask users to install it once as a direct dependency, so there's only one copy of @babel/runtime for the entire project. In some cases, your JS framework might already include @babel/runtime as a dependency.

Option 2: Include the minified JS file for use in the browser directly

<script src="dist/typesense.min.js"></script>

or via jsDelivr

<script src="https://cdn.jsdelivr.net/npm/typesense@1/dist/typesense.min.js"></script>

Usage

Read the documentation here for detailed examples: https://typesense.org/docs/api/

Tests are also a good place to know how the library works internally: test

Note: When using this library in a browser, please be sure to use an API Key that only allows search operations instead of the master API key. See doc/examples/server/keys.js for an example of how to generate a search only API key.

See Configuration.ts for a list of all client configuration options.

Examples

Here are some examples with inline comments that walk you through how to use the client: doc/examples

To run the examples, from the repo root:

npm run typesenseServer
node doc/examples/server/bulkImport.js

GatsbyJS Integration

If you use GatsbyJS for a framework, we have a plugin (that uses typesense-js behind the scenes) to automatically push your site data to Typesense when you build your site. Learn more here.

Firebase Integration

If you use Firebase, we have a Firebase extension (that uses typesense-js behind the scenes) to automatically push your Firestore data to Typesense. Learn more here.

Building UI components

Checkout the Typesense-InstantSearch.js (which uses typesense-js) for UI components you can use to quickly build powerful instant search experiences.

Compatibility

This table refers to server=>client compatibility. Newer versions of the client library maintain backwards compatibility with older versions of the server library.

Typesense Server	typesense-js
>= v0.26.0.rc38	>= v1.8.0
>= v0.25.0	>= v1.7.0
>= v0.24.0	>= v1.5.0
>= v0.23.0	>= v1.3.0
>= v0.21.0	>= v0.14.0
>= v0.20.0	>= v0.12.0
>= v0.19.0	>= v0.11.0
>= v0.18.0	>= v0.10.0
>= v0.17.0	>= v0.9.0
>= v0.16.0	>= v0.8.0
>= v0.15.0	>= v0.7.0
>= v0.12.1	>= v0.5.0
>= v0.12.0	>= v0.4.7
<= v0.11	<= v0.3.0

Development

After checking out the repo, run npm install to install dependencies. Then run npm test to run the linter and tests.

To release a new version, we use the np package:

$ npm install --global np
$ np

# Follow instructions that np shows you

Contributing

Bug reports and pull requests are welcome on GitHub at https://github.com/typesense/typesense-js.

typesense-js's Issues

Additional headers

Description

Our Typesense instance is located in distant server, and we are using Cloudflare Access, which require us to add custom headers through the request.

Expected Behavior

We should be able to add custom headers in the client configuration like:

new Typesense.Client({
    nodes: [{ host, port , protocol }],
    apiKey: 'key',
    headers: {
        'CF-Access-Client-Id': 'qwerty',
        'CF-Access-Client-Secret': 'xxx',
    },
});

Metadata

Typsense Version: 1.0.0-2 / 0.15.0

SearchClient doesn't return type-parameterized Collection

typesense-js/src/Typesense/SearchClient.ts

Lines 26 to 39 in 100d619

	collections(collectionName: string): Collection {
	// Nick: changed to less strict check, as null or an empty string would fail this statement
	if (!collectionName) {
	throw new Error(
	'Typesense.SearchClient only supports search operations, so the collectionName that needs to ' +
	'be searched must be specified. Use Typesense.Client if you need to access the collection object.'
	)
	} else {
	if (this.individualCollections[collectionName] === undefined) {
	this.individualCollections[collectionName] = new Collection(collectionName, this.apiCall, this.configuration)
	}
	return this.individualCollections[collectionName]
	}
	}

You can see here that collections(...) in SearchClient does not return a type-parameterized version of Collection. This is not similar to the normal Client which does allow a type parameter in the collections(...) function there.

In SearchClient, I think the collection(...) function signature should be like this:

collections<T extends DocumentSchema = {}>(collectionName: string): Collection<T> { 
  ...
}

I stumbled across this issue while trying to make an Angular module for TypeSense. I'd love to release a small Angular wrapper for TypeSense, but the inconsistency with type-generics is holding me back from making a release-ready version. (That's not mentioning that I've never really released a wrapper lib like what I'm working on before, it would be a joy to try)

I'm loving the lightness of TypeSense so far! I can tell that the JS/TS library here is rapidly evolving for the better, and I hope to help! Let me know if me opening a PR would be helpful.

Network Error React Native

// Results in Network Error --- (This exact request works fine on Node)

    const handleText = async (str?: string) => {
    const search = new Client({
      nodes: [{ host: 'localhost', port: 8108, protocol: 'http' }],
      apiKey: API_KEY,
    });

    let searchParameters = { q: str, query_by: 'title', };

    const res = await search
      .collections('roles')
      .documents()
      .search(searchParameters);
    setTerm(str || '');
}

// Works fine

const handleText = async (str?: string) => {
    setTerm(str || '');
    try {
      const res = await axios({
        url: `http://localhost:8108/collections/roles/documents/search?q=${
          str || '*'
        }&query_by=title&per_page=200`,
        method: 'get',
        headers: { 'X-TYPESENSE-API-KEY': API_KEY },
      });

      console.log({ ...res.data, hits: res?.data?.hits?.[0] });
      setHits(res?.data?.hits);
    } catch (err) {
      console.warn(err);
    }
  };

// Error

// Tried this with both Client and SearchClient

React Native Version 0.62 IOS, TypeSense 0.6.0

Error: Cannot find module

When requiring the NodeJS module typesense, I get this error:

Error: Cannot find module '@babel/runtime/helpers/interopRequireDefault'

I am now using dist/typesense instead.

Memory leak in the implementation of `RequestWithCache`.

Description

In the current implementation, a cache entry is only deleted when it is requested and it has expired. If the cache entry is never requested, it would never be deleted. Combined with the fact _responseCache has no limit on how many entries it can store, we've got a leak.

    if (cacheEntry) {
      if (Date.now() - cacheEntry.requestTimestamp < cacheResponseForSeconds * 1000) {
        // Cache entry is still valid, return it
        return Promise.resolve(cacheEntry.response)
      } else {
        // Cache entry has expired, so delete it explicitly
        delete this._responseCache[requestFunctionArgumentsJSON]
      }
    }

Steps to reproduce

1. Enable caching of results
2. Keep searching for stuff without ending the session, memory usage would steadily increase.

Expected Behavior

There should be an eviction policy in place to get rid of the unwanted cached entries and preferably an upper limit on how many results can be cached. I'd recommend implementing Least Recently Used algorithm.

Errors should return code

Currently errors do not seem to include the API error code, just a message.

port should be optional

Especially if typesense is behind a reverse proxy, the url does not necessarily have a url exposed.

An example setup:

https://example.com/typesense

proxies internally to the typesense docker container running at localhost:8108

prefix for multiple fields in search parameters is not possible

Description

According to the search parameters docs, we can define the behavior of prefix for multiple fields such as prefix=true,false,true.

However, this is not doable as prefix is defined as a boolean variable in the SearchParams.

I tried to add ?prefix=true in the query_by but that didn't work either.

Typescript build errors

Description

We have a Typescript project, built using Webpack + ts-loader. In the client SPA, I've imported Typesense the traditional way import * as Typesense from 'typesense' which seems to work well (typings come through). The code builds and runs fine, but at build time I'm getting the following errors, so our CI/CD pipeline stops :(

ERROR in /app/node_modules/typesense/lib/Typesense/Documents.d.ts
2:12-13
[tsl] ERROR in /app/node_modules/typesense/lib/Typesense/Documents.d.ts(2,13)
      TS1005: '=' expected.
ts-loader-default_3ef1a0ebb55d0532

ERROR in /app/node_modules/typesense/lib/Typesense/Documents.d.ts
2:32-36
[tsl] ERROR in /app/node_modules/typesense/lib/Typesense/Documents.d.ts(2,33)
      TS1005: ';' expected.
ts-loader-default_3ef1a0ebb55d0532

Expected Behavior

Should transpile without errors. Is there a different way I should be importing the browser client?

Metadata

Typsense Version:
"typesense": "^1.1.2"
"@babel/runtime": "^7.16.7" <-- we aren't running Babel, is it needed?

OS:
OSX BigSur 11.6

NPM package name

Probably should be "typesense-js" not "typesense" to limit confusion.

relative uri

Hi,

Is there any way to use the typesense client in a way that does not hard code the entire protocol? Namely, I would like to use relative url (eg /typesense) such that if the url changes the interaction would still work.

As of now I guess the solution would be to manually extract that information from the window.location, however that does not feel like a great option.

Any thoughts?

`SearchClient` ignores `sendApiKeyAsQueryParam=true` and send the api key in the query string

Description

When the SearchClient is instantiated with a ConfigurationsOptions object container sendApiKeyAsQueryParam=true the query string paramx-typesense-api-key still appears in the outgoing request.

Steps to reproduce

I have the following two snippets in a React app.

  const tsSearchClient = new TypesenseSearchClient({
    nodes: [
      {
        host: <typesene_host>, 
        port: <port>,
        protocol: 'http',
      },
    ],
    sendApiKeyAsQueryParam: false,
    apiKey: <typesense_api_key',
    connectionTimeoutSeconds: 2,
  });

let response = await typesenseClient.multiSearch.perform(searchReq(), commonSearchParams(queryInput));

Pull up Chrome dev tools, network tab. You'll be able to see the param in the request URL.

Expected Behavior

The default appears to be sendApiKeyAsQueryParam=false in the implementation which makes sense to me. Unless explicitly set to true I expect the key to NOT appear in the request URL.

Actual Behavior

Regardless of the attribute being set to true, false, or not present the key appear in the request URL.

Metadata

Typsense Version: 1.1.3-0

OS:
macOS Monterey 12.1
Chrome: Version 96.0.4664.110 (Official Build) (x86_64)

Use a configuration class rather than an object

Similar to: typesense/typesense-ruby#9
(just to ensure that all clients maintain the same convention)

Publish to NPM

TypeScript for Typesense?

Hello, how do I get typescript support when importing this library? It isn't critical but would be nice.

Proxy in production

I am have a simple express, react, typesense set up in a docker-compose environment. I have been using localhost:8108 to access typesense and it works fine. But when I change to production mode, react cannot get data from typesense because localhost:8108 is no longer valid on the client side. I want to know how can I proxy the request to the client.

GenerateScopedSearchKeyParams missing `collection`

Description

I think the collection definition is probably missing here

https://github.com/typesense/typesense-js/blob/master/src/Typesense/Keys.ts#L12

Steps to reproduce

Expected Behavior

Can use collection to generate a scoped search key

Actual Behavior

Type error

Metadata

Typsense Version:

OS:

Search result edge case

Description

This isn't a bug, but a specific edge case that I'm wondering if I missed something in my configuration or if there's something I can do to improve my results.

I have a collection with data linked from Firestore hosted on typesense cloud and am running a nodejs script on my local computer via the javascript library to search for results. Everything works very well except when I search for strings that "look" like a number (i.e. could implicitly casted as integers), but needs to be a string (an example would be a phone number).

Here's some tests I run:
My stored string is "z123" and search "123". Doesn't work
My stored string is "z123" and search "z123". Works
My stored string is "123z" and search "123". Works

While my use case above might not be easy to understand, I tried some other examples with a phone number that will make more sense.

My stored string is "+1.123.456.7890" and search "123". Doesn't work
My stored string is "1.123.456.7890" and search "123". Doesn't work
My stored string is "456.123.7890" and search "123". Doesn't work
My stored string is "123.456.7890" and search "123". Works

Steps to reproduce

See the tests above. Here is my code from nodejs

searchResults = await typesense.collections('my_collection').documents().search({
    q: '123',
    query_by: 'my_field'
})

Expected Behavior

To return results when searching via strings that could be implicitly casted as integers

Actual Behavior

Return results from the examples above

Metadata

Typsense Version: v0.22.1

OS: typesense hosted cloud

How To support array of objects search field types ?

I have a json file which contains an array of objects and I want to crearte a schema for that so how to do it ?

Types for import API options missing batch_size

Description

Type for import API options missing batch_size. As described here it is clearly an option and it does work (at the very least doesn't err) for me.

Steps to reproduce

// try to import some documents like so
const results = await this.typesenseClient
      .collections(this.collectionsName)
      .documents()
      .import(documents, { action: 'upsert', batch_size: 200 });

Expected Behavior

I expect the code above to not err at transpilation time.

Actual Behavior

I get TS2769: No overload matches this call.   Overload 1 of 2, '(documents: string, options?: DocumentWriteParameters): Promise<string>', gave the following error.     Argument of type 'TypeSenseRecord[]' is not assignable to parameter of type 'string'.   Overload 2 of 2, '(documents: {}[], options?: DocumentWriteParameters): Promise<ImportResponse[]>', gave the following error.     Argument of type '{ action: "upsert"; batch_size: number; }' is not assignable to parameter of type 'DocumentWriteParameters'.       Object literal may only specify known properties, and 'batch_size' does not exist in type 'DocumentWriteParameters'..

Metadata

Typsense Version: 0.22.2

OS: Windows 11 (10.0.22000.556)

Feedback on TS support

Description

Should these deprecated properties be required?

Is index supposed to be required on Node?

Setup greenkeeper-lockfile

Based on Greenkeeper's instructions from here: #6

[Feature request] Cache search operations

With several other search clients (e.g., Algolia's), search results are cached on the client. This way, users don't need to perform unnecessary search requests (e.g., when deleting the last character in their query). This also reduces latency for repeated searches to 0.

I've personally implemented this on my end with the following pattern:

const cachedSearches = {}

export const search = async (options) => {
    const optionsString = JSON.stringify(options)
    if (typeof cachedSearches[optionsString] !== 'undefined') return cachedSearches[optionsString]
    const response = await client.collection('collection').documents.search(options)
    // if statement reduces possible race conditions
    if (typeof cachedSearches[optionsString] === 'undefined') cachedSearches[optionsString] = response
    return response
}

My full code is a bit more complex, mainly making sure edge cases (e.g., options appearing in a different order) are still cached correctly. But this is the main idea.

Under the hood, Typesense could use the final search URL as the keys for the cached searches. That way, all collections can be cached with the same cache.

group_by is keyof T while in documentation is it a string

Description

I was trying to group a search by a field that is defined as a facet when I had an error in the editor:

Argument of type '{ q: string; query_by: string; group_by: string; group_limit: string; }' is not assignable to parameter of type 'SearchParams<{}>'.
  Types of property 'group_by' are incompatible.
    Type 'string' is not assignable to type 'never'.ts(2345)

I am using typescript.

Steps to reproduce

1. Create a search client.
2. Create a searchParam object:

	let searchParameters = {
		q: req.params.location,
		query_by: 'city',
		group_by: 'state',
		group_limit: '1',
	};

Expected Behavior

It should not raise any errors when using Typescript.

Actual Behavior

Error:

Argument of type '{ q: string; query_by: string; group_by: string; group_limit: string; }' is not assignable to parameter of type 'SearchParams<{}>'.
  Types of property 'group_by' are incompatible.
    Type 'string' is not assignable to type 'never'.ts(2345)

Documentation

Search Parameters - Grouping

Metadata

Typsense Version: 1.1.2

OS: Mac M1Pro

'expires_at' is missing in the parameter of'generateScopedSearchKey'

Description

According to the official website, you can specify'expires_at' when creating a restricted key.
https://typesense.org/docs/0.21.0/api/api-keys.html#generate-scoped-search-key

However, VScode prints a warning because there is no'expires_at' in the TypeScript type definition.

Steps to reproduce

write code

import * as Typesense from 'typesense'
async function makeLimitKey () {
  const client = new Typesense.Client({
    nodes: [{
      host: 'samplehost_xxxxx',
      port: 443,
      protocol: 'https'
    }],
    apiKey: 'apikey_secret_xxxxxx',
    connectionTimeoutSeconds: 2
  })
  
  const dangerKey = await client.keys().create({
    'description': 'xxxxxx',
    'actions': ['documents:search'],
    'collections': ['*']
  })
  if (dangerKey.value === undefined) {
    return false
  }
  const expiresAt = dayjs().add(30, 'day').unix()
  
  const limitedKey = client.keys().generateScopedSearchKey(
    dangerKey.value,
    {
      'filter_by': 'roomId:dummyData',
      'expires_at': expiresAt // <= here warning
    }
  )
}

Expected Behavior

No warning is displayed

Actual Behavior

Warning is displayed

Metadata

VSCode 1.16.0
TypeScript 4.2.4

Typsense Version:
Version 1.0.0

OS:

Error: getaddrinfo ENOTFOUND master (Mac prebuilt binary)

Just trying to follow the getting started tutorial. I'm sure I'm making a terribly stupid error somewhere, but I can't figure out where.

My Code

const Typesense = require("typesense");

const searchClient = new Typesense.Client({
  masterNode: {
    host: "master",
    port: 8108,
    protocol: "http",
    apiKey: "foobar"
  }
});

let showsSchema = {
  name: "shows",
  fields: [
    { name: "name", type: "string" },
    { showName: "name", type: "string" }
  ]
};

searchClient
    .collections()
    .create(showsSchema)
    .then(console.log)
    .catch(console.log);

Error

Error: getaddrinfo ENOTFOUND master
    at /Users/lustig/Library/Mobile Documents/com~apple~CloudDocs/Development/CLIENTS/TCT/SERVER/node_modules/typesense/lib/Typesense/Typesense/ApiCall.js:66:5

TypeSense server output

Error: getaddrinfo ENOTFOUND master
    at /Users/lustig/Library/Mobile Documents/com~apple~CloudDocs/Development/CLIENTS/TCT/SERVER/node_modules/typesense/lib/Typesense/Typesense/ApiCall.js:66:5

Please let me know if you require any more information

Export documents as stream

Description

documents().export() currently fetches the entire API response, stores it in a string and returns the string. When exporting large collections with gigabytes of data, a better memory-efficient approach would be to expose a stream object so that users can then operate on the stream and pipe it directly to a file for eg.

To maintain backward compatibility, we can introduce a new method documents().exportStream() that returns a ReadStream, that's returned by axios when responseType: 'stream' is set.

ObjectNotFound error

getting this error when i try to create a document.
curl localhost:8108/health returns {'ok': true} but localhost:8108/ returns { "message": "Not Found"}

/Users/manav/projects/fluxroom-beta/node_modules/typesense/lib/Typesense/ApiCall.js:366
var customErrror = new CustomErrorKlass(errorMessage);
^

ObjectNotFound [Error]: Request failed with HTTP code 404 | Server said: Not Found
at ApiCall._customErrorForResponse (/Users/manav/projects/fluxroom-beta/node_modules/typesense/lib/Typesense/ApiCall.js:366:26)
at ApiCall._callee$ (/Users/manav/projects/fluxroom-beta/node_modules/typesense/lib/Typesense/ApiCall.js:178:70)
at tryCatch (/Users/manav/projects/fluxroom-beta/node_modules/regenerator-runtime/runtime.js:63:40)
at Generator.invoke [as _invoke] (/Users/manav/projects/fluxroom-beta/node_modules/regenerator-runtime/runtime.js:293:22)
at Generator.next (/Users/manav/projects/fluxroom-beta/node_modules/regenerator-runtime/runtime.js:118:21)
at asyncGeneratorStep (/Users/manav/projects/fluxroom-beta/node_modules/@babel/runtime/helpers/asyncToGenerator.js:3:24)
at _next (/Users/manav/projects/fluxroom-beta/node_modules/@babel/runtime/helpers/asyncToGenerator.js:25:9)
at processTicksAndRejections (node:internal/process/task_queues:94:5) {
httpStatus: 404
}

How to retrieve all the documents from a collection?

I am doing this
client
.collections("queries")
.documents()
.search({
q: "*",
query_by: "query_title"
});
But it is giving zero hits

NPM Release does not contain updated 0.9.0 code

Description

The version installed by npm is outdated and doesn't include the latest code in lib. For example, the new documents().delete() method is undefined. NPM only includes the lib directory, no source or anything else.

Steps to reproduce

npm install [email protected]

Make a smaller Bundle Size?

TypeSense JS for use in a React, or React Native apps has a massive bundle size of 1mb which is minified to 463kb. This is way too high to be used in production and adds ~3 seconds to page loading times on lower bandwidth connections. Like mobile phones or laptops on Public WiFi.

Competitors in the scene have JS packages that are less that 30kb minified and are Tree-Shakeable. This isn't and I was wondering if there are any plans to substantially improve this issue?

Here Is a link for clarity.

TypeSense JS also has 3 dependencies which are large in themselves, the biggest is
@babel/runtime, & Axios.

String `undefined` being attach to collection name.

Description

I am working on a Sveltekit product. On production only, the API call attaches a string undefined as part of the query.

this is the code:

const result = await client.collections('cards').documents().search({
      q: term,
      prefix: true,
      num_typos: '1',
      query_by: 'firstName,lastName',
      per_page: 10,
      sort_by: 'sort1:asc,sort2:asc',
      page: currentPage,
    });

Version: "typesense": "^1.1.3-0"

Request Cancellation

Description

I'd like to be able to abort requests using the AbortController API...

https://developer.mozilla.org/en-US/docs/Web/API/AbortController

Steps to reproduce

This is a feature request not a bug.

Expected Behavior

There'd be some way to pass an AbortController to each underlying axios call in order to cancel an ongoing request before another is made. This not only saves resources but also helps to avoid race conditions where an older request comes back after a newer one (e.g. when typing fast).

https://axios-http.com/docs/cancellation

Actual Behavior

As far as I can tell, there's no way to do this as of right now using things like .collections(...).documents().search(...). I think we'd need to add an optional second argument to anything using ApiCall

Metadata

Typsense Version: SDK v0.12.1 / Server v0.22.2

OS: MacOS

TypeError: client.collections(...).documents(...).retrieve is not a function

Description

Calling client.collections('some-collection').documents('some-doc-id').retrieve() throws the following error:

UnhandledPromiseRejectionWarning: TypeError: client.collections(...).documents(...).retrieve is not a function

Steps to reproduce

Just call the method like specified in the Typesense docs. My code looks like the following:

export const getDoc = async <T = any>(
    client: Typesense.Client,
    collection: SearchCollection,
    documentId: string
) => client.collections<T>(collection).documents(documentId).retrieve();

Expected Behavior

This method should exist and should get the document associated with the documentId that was passed as outlined in the retrieve a document section of the docs

Actual Behavior

You get a "retrieve() is not a function" type error.

Metadata

• Typesense v0.22.1
• Typesense-JS v1.0.2
• Node v14.17.0
• Npm v8.3.0

OS:
Windows 10

TS typing issue for OverrideCreateSchema

Description

The TS typings for the OverrideCreateSchema interface appear to be incorrect:

typesense-js/src/Typesense/Overrides.ts

Lines 7 to 21 in 15ae8b6

	export interface OverrideCreateSchema {
	rule: {
	query: string
	match: 'exact' | 'contains'
	}
	filter_by?: string
	remove_matched_tokens?: boolean
	includes?: [
	{
	id: string
	position: number
	}
	]
	excludes?: [{ id: string }]
	}

Specifically, the includes and excludes properties are typed as single-element tuples, rather than arrays.

Expected Behavior

I would expect the type to be:

  includes?: Array<{
      id: string
      position: number
    }>
  excludes?: Array<{ id: string }>

Actual Behavior

Attempting to assign an array will result in the TS error:

TS2322: Type '{ id: string; }[]' is not assignable to type '[{ id: string; }]'.   Target requires 1 element(s) but source may have fewer.

Metadata

Typsense Version: 1.2.1

OS: Windows 10

Feature request: make client http proxy friendly, omit port if undefined

I'm using Typesense through a HTTP proxy at a specific path (example: https://example.com/typesense). In this case it's not desirable that a port is required when specifying such a node:

typesense-js/src/Typesense/ApiCall.js

Line 179 in ed1b22d

return `${node.protocol}://${node.host}:${node.port}${node.path}${endpoint}`

Could this be accounted for in an upcoming release?

Typesense CLI

Description

This is not a bug report!!!
I have recently set up a self-hosted typesense server and have been interacting with it using a Typesense js client. I was thinking of potentially taking this further and producing a typesense cli that would allow you to manage the server directly from the terminal. Is this something you would be interested in?

Metadata

Typesense Version: 0.13.0:

OS: ubuntu 21.04:

allowed types for SearchParams['query_by'] don't match documentation

Description

according to docs, SearchParams['query_by'] accepts string arrays, however the type-definitions provided are limited to string only.

typesense module for Deno

It would be awesome if there was a Deno compatible typesense module.

You can almost directly import * as Typsense from src/Typesense.ts but there are some issue:

• Deno requires all local imports to have explict .ts extensions
  So instead of import ApiCall from './ApiCall' use import ApiCall from './ApiCall.ts'
  Also Errors, instead of import { ObjectNotFound } from './Errors' use import { ObjectNotFound } from './Errors/ObjectNotFound.ts'

• Keys.ts needs modified to use the hmac and base64 modules from Deno:
  Change: import { createHmac } from 'crypto' to

import {hmac} from "https://deno.land/x/[email protected]/mod.ts";
import {encode as base64Encode} from "https://deno.land/[email protected]/encoding/base64.ts"

And change lines

const digest = Buffer.from(createHmac('sha256', searchKey).update(paramsJSON).digest('base64'))
return Buffer.from(rawScopedKey).toString('base64')

to

const digest = hmac("sha256", searchKey, paramsJSON, "utf8", "base64")
return base64Encode(rawScopedKey);

• Configuration.js needs a loglevel replacement. I just used a stub:

this.logger =
{
	trace : v => console.log(v),
	debug : v => console.log(v),
	info : v => console.info(v),
	warn : v => console.warn(v),
	error : v => console.error(v),
	log : v => console.log(v),
	setLevel : () => {}
}

This is where I gave up because there are at least 3 remaining issues:

• Documents.ts needs a ReadStream Deno equilivant

• All of axios needs replaced in ApiCall.ts Maybe you could use the Deno version https://github.com/roonie007/axiod but much more ideally you could use the built in Deno fetch support

• Lastly, there may be other 'nodejs' specific APIs being used in the code that I didn't look for

Backend InstantSearch Support

Feature Request

Thank you for this awesome library. Please look at this issue as a feature request.

Description

I was trying to follow Algolia's Backend InstantSearch guide but I faced some issue. I finally managed to make it work but I had to add extra fields like nbHits, hitsPerPage, nbPages, objectID to the returned results to make it work.

const Typesense = require("typesense");
const typesense = new Typesense.Client({
  nodes: [{ host: "*****.ngrok.io", port: "443", protocol: "https" }],
  apiKey: "xyz",
});
export default async function handler({ body }, res) {
  const { requests } = body;
  const typesenseResults = await typesense.multiSearch.perform({
    searches: requests,
  });
  const results = {
    ...typesenseResults,
    results: typesenseResults.results.map((result) => ({
      ...result,
      nbHits: result.found,
      nbPages: result.out_of,
      hitsPerPage: 9,
      hits: result.hits.map((hit) => ({
        ...hit,
        objectID: hit.document.id,
        document: { ...hit.document },
      })),
    })),
  };
  res.status(200).send(results);
}

Are you planning to make an adaptor for the TypesenseClient library to return the same results as Algolia's client library does?

algoliaClient.searchForFacetValues(requests);
algoliaClient.search(requests);

Use Case

I use this feature to add and remove unnecessary data for performance reasons. As well as in house built-in authorization to prevent an unauthenticated user to access certain fields.

How to use scoped search keys for search?

Description

I'm trying to use scoped search keys to limit the content that users can access. However, I don't get the key to work for search operations. What is the correct way of implementing this?

Steps to reproduce

Create a scoped search key:

client
  .keys()
  .create({
    description: "Admin key",
    actions: ["documents:search"],
    collections: ["items"],
  })
  .then((key) => {
    const searchKey = client.keys().generateScopedSearchKey(key.value, {
      filter_by: `workpaceId:=1`,
      expires_at: 64723363199,
    });
    console.log(searchKey);
  })

Create a new client using this key as the API key:

const searchClient = new Typesense.Client({
  nodes: [
    {
      host: "HOSTNAME.typesense.net",
      port: "443",
      protocol: "https",
    },
  ],
  apiKey: "SCOPED_SEARCH_KEY",
  connectionTimeoutSeconds: 2,
})

Use the client to search among items:

const searchParameters = {
  q: "city",
  query_by: "tags",
  sort_by: "popularity:desc",
  filter_by: "workspaceId:=1",
};

searchClient
  .collections("items")
  .documents()
  .search(searchParameters)
  .then((searchResults) => {
    console.log(searchResults)
  })
  .catch((err) => console.error(err))

Expected Behavior

Documents should be logged. I get the following when initializing the client using the admin key instead of the scoped search key:

{
  facet_counts: [],
  found: 1,
  hits: [
    {
      document: {
        id: '1',
        popularity: 1,
        tags: [ 'woman', 'man', 'city', 'traffic' ],
        workspaceId: '1'
      },
      highlights: [
        {
          field: 'tags',
          indices: [ 2 ],
          matched_tokens: [ [ 'city' ] ],
          snippets: [ '<mark>city</mark>' ]
        }
      ],
      text_match: 130816
    }
  ],
  out_of: 3,
  page: 1,
  request_params: { collection_name: 'items', per_page: 10, q: 'city' },
  search_time_ms: 0
}

Actual Behavior

404 error:

ObjectNotFound [Error]: Request failed with HTTP code 404 | Server said: Not Found
    at ApiCall._customErrorForResponse (/Users/frithiofekstrom/github.com/fekstr/typesense-test/node_modules/typesense/lib/Typesense/Typesense/ApiCall.js:238:26)
    at ApiCall._callee$ (/Users/frithiofekstrom/github.com/fekstr/typesense-test/node_modules/typesense/lib/Typesense/Typesense/ApiCall.js:121:38)
    at tryCatch (/Users/frithiofekstrom/github.com/fekstr/typesense-test/node_modules/regenerator-runtime/runtime.js:63:40)
    at Generator.invoke [as _invoke] (/Users/frithiofekstrom/github.com/fekstr/typesense-test/node_modules/regenerator-runtime/runtime.js:293:22)
    at Generator.next (/Users/frithiofekstrom/github.com/fekstr/typesense-test/node_modules/regenerator-runtime/runtime.js:118:21)
    at asyncGeneratorStep (/Users/frithiofekstrom/github.com/fekstr/typesense-test/node_modules/@babel/runtime/helpers/asyncToGenerator.js:3:24)
    at _next (/Users/frithiofekstrom/github.com/fekstr/typesense-test/node_modules/@babel/runtime/helpers/asyncToGenerator.js:25:9)
    at processTicksAndRejections (internal/process/task_queues.js:97:5) {
  httpStatus: 404
}

Metadata

Typesense Version: 0.19.0

Typesense JS version: 0.11.2

OS: Typesense running on Typesense Cloud. JS running on MacOS Big Sur.

Timeout with Node Version <= 16

Description

I set up my own self hosted typesense instance on a vps. Now when I use typesense-js with node version 16 or less I will always get a timeout. If I use Node 17 everything works fine. I guess it has to do something with my server setup or dns setup, since everything works fine when I start a local docker container. When I check the health endpoint I can curl it just fine.
Does anyone else had this issue and knows what to do?

The argument type of client.keys () is a string.

Description

Official sample code
https://typesense.org/docs/0.21.0/api/api-keys.html#delete-api-key

According to this, the argument of client.keys () is a number.
But TypeScript type definitions require strings

Steps to reproduce

Expected Behavior

According to the operation manual, it is correct to pass the numerical value.

Actual Behavior

I get a warning when I pass a number.

Metadata

Typsense Version:
typesense 1.0.2

OS:

`SearchResponseHit.highlights` and `SearchResponseFacetCountSchema.counts` has invalid type definitions.

Description

SearchResponseHit.highlights and SearchResponseFacetCountSchema.counts has invalid type definitions. They should be in array format.

Metadata

Typsense Version: 1.0.3-4

Could not load source file "../src/Typesense.ts" in source map of "node_modules/typesense/lib/Typesense.js"

Description

Running parcel build shows these warnings:

00:21:36.587 | $ parcel build index.html --public-url https://airbnb-geosearch.typesense.org
-- | --
00:21:45.993 | ⚠️  Could not load source file "../src/Typesense.ts" in source map of "node_modules/typesense/lib/Typesense.js".
00:21:46.657 | ⚠️  Could not load source file "../src/Typesense.ts" in source map of "node_modules/typesense/lib/Typesense.js".
00:21:53.342 | ⚠️  Could not load source file "../../src/Typesense/Client.ts" in source map of "node_modules/typesense/lib/Typesense/Client.js".
00:21:53.347 | ⚠️  Could not load source file "../../src/Typesense/SearchClient.ts" in source map of "node_modules/typesense/lib/Typesense/SearchClient.js".
00:21:56.321 | ⚠️  Could not load source file "../../src/Typesense/Aliases.ts" in source map of "node_modules/typesense/lib/Typesense/Aliases.js".
00:21:56.322 | ⚠️  Could not load source file "../../src/Typesense/Alias.ts" in source map of "node_modules/typesense/lib/Typesense/Alias.js".
00:21:56.322 | ⚠️  Could not load source file "../../src/Typesense/Keys.ts" in source map of "node_modules/typesense/lib/Typesense/Keys.js".
00:21:56.323 | ⚠️  Could not load source file "../../src/Typesense/Key.ts" in source map of "node_modules/typesense/lib/Typesense/Key.js".
00:21:56.324 | ⚠️  Could not load source file "../../src/Typesense/Debug.ts" in source map of "node_modules/typesense/lib/Typesense/Debug.js".
00:21:56.338 | ⚠️  Could not load source file "../../src/Typesense/Configuration.ts" in source map of "node_modules/typesense/lib/Typesense/Configuration.js".

Steps to reproduce

1. Clone https://github.com/typesense/showcase-airbnb-geosearch
2. yarn install
3. parcel build

Expected Behavior

No warnings are logged

Actual Behavior

The warnings above are logged

Metadata

Typsense.js Version: 1.0.3-2

Setup CircleCI

protocol should take trailing `:`

the protocol format for the URL per the window.location object, and URL spec includes a trailing :

window.location.protocol \\ 'http:' or 'https:'

perhaps could 'sanitize' the protocol on construction to remove a trailing : if it exists

Import errors have no message

ImportError throws with no message. It looks like this line should be super(message)

typesense-js/src/Typesense/Errors/ImportError.ts

Lines 6 to 9 in 3d7c270

	constructor(message, importResults) {
	super()
	this.importResults = importResults
	}

I was thoroughly confused about what was going wrong in my import because I couldn't see this (very helpful) error message:

typesense-js/src/Typesense/Documents.ts

Lines 227 to 232 in 7cc70b4

	throw new ImportError(
	`${resultsInJSONFormat.length - failedItems.length} documents imported successfully, ${
	failedItems.length
	} documents failed during import. Use \`error.importResults\` from the raised exception to get a detailed error reason for each document.`,
	resultsInJSONFormat
	)

Error for missing header is sent when passing a bad API key

Description

When using the API as described by the docs, I am getting the a missing heading error

Steps to reproduce

const getTypeSenseClient = () => {
  
  const secret: { readKey; adminKey; host } = typeSenseSecret()
  return new Client({
    'nodes': [{
      'host': secret.host,
      'port': '443',
      'protocol': 'https'
    }],
    'apiKey': secret.adminKey,
    'connectionTimeoutSeconds': 2
  })
}

const importMessages = async () => {
  await connectDB(false, false)
  
  const messageRepo = getRepository(Message)
  const typesenseClient = getTypeSenseClient()
  
  const fetchedMessages = await messageRepo.find({
    select: ["timestamp", "type", "content", "senderId", "id", "chatId", "deleted"],
    where: {type: "draft"}
  })
  
  const parsedMessages = fetchedMessages.filter((msg) => {
    return JSON.parse(msg.content) !== null
  }).map( (msg) => {
    const messageContent = JSON.parse(msg.content).blocks.reduce((acc, current) => {
      return acc + current.text + "\n"
    }, "")
    
    return {
      content: messageContent,
      timestamp: new Date(msg.timestamp).getTime(),
      senderId: msg.senderId,
      type: msg.type,
      upload: 'false',
      chatId: msg.chatId,
      id: msg.id,
      deleted: msg.deleted
    }
  })
  
  typesenseClient.collections('messages').documents().import(parsedMessages, {action: 'create'})
}

Results in the following error

/Users/sean/workspace/pesto-repos/pesto/infra-2/node_modules/typesense/lib/Typesense/Typesense/ApiCall.js:261
    const customErrror = new CustomErrorKlass(errorMessage)
                         ^
RequestUnauthorized [Error]: Request failed with HTTP code 401 | Server said: Forbidden - a valid `x-typesense-api-key` header must be sent.
    at ApiCall._customErrorForResponse (/Users/sean/workspace/pesto-repos/pesto/infra-2/node_modules/typesense/lib/Typesense/Typesense/ApiCall.js:261:26)
    at ApiCall._callee$ (/Users/sean/workspace/pesto-repos/pesto/infra-2/node_modules/typesense/lib/Typesense/Typesense/ApiCall.js:137:38)
    at tryCatch (/Users/sean/workspace/pesto-repos/pesto/infra-2/node_modules/regenerator-runtime/runtime.js:63:40)
    at Generator.invoke [as _invoke] (/Users/sean/workspace/pesto-repos/pesto/infra-2/node_modules/regenerator-runtime/runtime.js:294:22)
    at Generator.next (/Users/sean/workspace/pesto-repos/pesto/infra-2/node_modules/regenerator-runtime/runtime.js:119:21)
    at asyncGeneratorStep (/Users/sean/workspace/pesto-repos/pesto/infra-2/node_modules/@babel/runtime/helpers/asyncToGenerator.js:3:24)
    at _next (/Users/sean/workspace/pesto-repos/pesto/infra-2/node_modules/@babel/runtime/helpers/asyncToGenerator.js:25:9)
    at processTicksAndRejections (node:internal/process/task_queues:94:5) {
  httpStatus: 401
}

Typsense Version: 0.15.0

OS: mac

Too many properties to enumerate when calling documents().import()

Reported on Slack.

Request #1626169731197: Request to Node 0 failed due to "undefined Too many properties to enumerate"
Request #1626169731197: Sleeping for 0.1s and then retrying request...
Request #1626169731197: Request to Node 0 failed due to "undefined Too many properties to enumerate"
Request #1626169731197: Sleeping for 0.1s and then retrying request...
RangeError: Too many properties to enumerate
    at Function.keys (<anonymous>)
    at ApiCall._callee$ (/home/<>/Documents/dt-site-search/node_modules/typesense/lib/Typesense/ApiCall.js:155:46)
    at tryCatch (/home/<>/Documents/dt-site-search/node_modules/regenerator-runtime/runtime.js:63:40)
    at Generator.invoke [as _invoke] (/home/<>/Documents/dt-site-search/node_modules/regenerator-runtime/runtime.js:293:22)
    at Generator.next (/home/<>/Documents/dt-site-search/node_modules/regenerator-runtime/runtime.js:118:21)
    at asyncGeneratorStep (/home/<>/Documents/dt-site-search/node_modules/@babel/runtime/helpers/asyncToGenerator.js:3:24)
    at _next (/home/<>/Documents/dt-site-search/node_modules/@babel/runtime/helpers/asyncToGenerator.js:25:9)

This happens due to a JS limitation: https://stackoverflow.com/questions/9282869/are-there-limits-to-the-number-of-properties-in-a-javascript-object. This also happens for large arrays of objects for some reason.

Instead of calling JSON.stringify on the entire array of objects here, we could split large arrays into smaller ones, then call JSON.stringify on them individually and then concat them together. So users of the client library don't have to do this themselves.