danitseitlin/redis-modules-sdk-ts

A Software development kit for easier connection and execution of Redis Modules commands.

A Software development kit for easier connection and execution of Redis Modules commands

A bit about us 💬

An open source SDK of all the available Redis modules, built with TypeScript for better typing experiences and better usages.

Benefits ⚡ 🙊

1. All in 1, all Redis modules are covered in 1 project
2. Amazing 🔥 typing experiences, all types are well documented and easy to use!
3. Easy to find your Redis command, each Redis module command has a referenced function in it's class!

Let's get started 📝

Installing latest version:

npm install redis-modules-sdk@latest

Versions & Releases

• A list of existing versions can be found here
• A list of releases will be found here

Documentation 📖

Come and read our documentation here before starting

(RAIO) Redis "All in One"! 😱

A class built for integrating more than one Redis module without creating more than one class!

const client = new Redis(....);
await client.connect();
await client.ai_module_tensorset(...);
await client.ris_module_add(...);
await client.disconnect();

All modules are supported! 🔥

Sandbox Playground

Want to play around with the code before installing it? Feel free to do so here

Supported modules 🕶️

• ReJSON
• RedisTimeSeries (RTS)
• RediSearch
• RedisGraph
• RedisGears
• RedisBloom
• RedisAI
• RedisIntervalSets

Contributing 🙌

Interested in contributing? awesome! start with reading our guidelines here

Supporters 👐

Forkers 🍴

redis-modules-sdk-ts's Issues

RedisTimeSeries: Support TS.MRANGE/MREVRANGE GROUPBY <label> REDUCE <reducer>

The following PR RedisTimeSeries/RedisTimeSeries#617 added support for multi-series aggregations to RedisTimeSeries.

It adds GROUPBY <label> REDUCE <reducer> to TS.MRANGE/TS.MREVRANGE. It accepts SUM, MIN, MAX reducers and works with unaligned series.

Sample request:

TS.MRANGE 1451679382646 1451682982646 WITHLABELS 
AGGREGATION MAX 60000 
FILTER measurement=cpu 
      fieldname=usage_user 
      hostname=(host_9,host_3,host_5,host_1,host_7,host_2,host_8,host_4)
GROUPBY hostname REDUCE MAX

Reply labels array structure

Labels:

• <label>=<groupbyvalue>
• __reducer__=<reducer>
• __source__=key1,key2,key3

---

Sample output:

127.0.0.1:6379> ts.add ts1 1 90 labels metric cpu name system
(integer) 1
127.0.0.1:6379> ts.add ts1 2 45 labels metric cpu name system
(integer) 2
127.0.0.1:6379> ts.add ts2 2 99 labels metric cpu name user
(integer) 2
127.0.0.1:6379> ts.add ts3 2 02 labels metric cpu name system
(integer) 2

MAX reducer sample output

127.0.0.1:6379> ts.mrange - + withlabels filter metric=cpu groupby name reduce max
1) 1) "name=system"
   2) 1) 1) "name"
         2) "system"
      2) 1) "__reducer__"
         2) "max"
      3) 1) "__source__"
         2) "ts1,ts3"
   3) 1) 1) (integer) 1
         2) 90
      2) 1) (integer) 2
         2) 45
2) 1) "name=user"
   2) 1) 1) "name"
         2) "user"
      2) 1) "__reducer__"
         2) "max"
      3) 1) "__source__"
         2) "ts2"
   3) 1) 1) (integer) 2
         2) 99

MIN reducer sample output

127.0.0.1:6379> ts.mrange - + withlabels filter metric=cpu groupby name reduce min
1) 1) "name=system"
   2) 1) 1) "name"
         2) "system"
      2) 1) "__reducer__"
         2) "min"
      3) 1) "__source__"
         2) "ts1,ts3"
   3) 1) 1) (integer) 1
         2) 90
      2) 1) (integer) 2
         2) 2
2) 1) "name=user"
   2) 1) 1) "name"
         2) "user"
      2) 1) "__reducer__"
         2) "min"
      3) 1) "__source__"
         2) "ts2"
   3) 1) 1) (integer) 2
         2) 99

TimeSeries Add GROUPBY and REDUCE support

Expected argument extension to TS.MRANGE: GROUPBY <label> REDUCE <reducer>

TS.MRANGE 1451679382646 1451682982646 WITHLABELS 
AGGREGATION MAX 60000 
FILTER measurement=cpu 
      fieldname=usage_user 
      hostname=(host_9,host_3,host_5,host_1,host_7,host_2,host_8,host_4)
GROUPBY hostname REDUCE MAX

Return Value

Array-reply, specifically it should include the following labels:

Labels:

• <label>=<groupbyvalue>
• __reducer__=<reducer>
• __source__=key1,key2,key3

Strict ESlinting more

Add support for RediSearch 2.2 JSON support

Features were added specific to JSON indexing:

On FT.CREATE:

• ON JSON
• identifier AS Attribute

e.g.
FT.CREATE idx ON JSON PREFIX 1 doc: SCHEMA $.product.label AS label TEXT SORTABLE $.product.stock AS stock NUMERIC SORTABLE

On FT.SEARCH:

• RETURN {jsonPath} AS {property}
  e.g.
  FT.SEARCH products * RETURN 2 $.product.label AS label $.reviews.content AS reviews

RTS - Add missing testing coverage

There is a need to fill in a missing gap of tests for commands: TS.RANGE, TS.REVRANGE, TS.MRANGE, TS.MREVRANGE
The tests exist, but are not working today. The need is to fix them, not add development and coverage.

Reuse the same connection to work with multiple Redis modules

I am using RedisGears and RedisTimeSeries modules for one of my databases. I would like to reuse the same connection to run RedisGears and RedisTimeSeries commands and don't worry about managing multiple connections for each module.

Similar to:

r = new Redis()
r.ftSearch()
r.graphQuery()
r.tsRange()

Thank you.

Update AI.ScriptExecute command as in docs

RediSearch (FT.SEARCH): ReplyError: Unknown argument

Hello!

I'm using this library to search my rediSearch database. I need some special parameters in my query so I added them to my function call like so:

rediSearchClient.search(
   'TrainsIndex',
    `12*`,
    {
        inFields: {
            num: 2,
            field: "tags number"
         },
        highlight: {
            tags: [{
                open: "<b>",
                close: "</b>"
            }],
        }
    }
);

But this query fails with the following error:

UnhandledPromiseRejectionWarning: Error: Redisearch class (FT.SEARCH): ReplyError: Unknown argument `TAGS` at position 5 for <main>
at Redisearch.handleError (/src/node_modules/redis-modules-sdk/lib/modules/module.base.js:76:19)
at Redisearch.<anonymous> (/src/node_modules/redis-modules-sdk/lib/modules/module.base.js:65:29)
at Generator.throw (<anonymous>)
at rejected (/src/node_modules/redis-modules-sdk/lib/modules/module.base.js:6:65)
at processTicksAndRejections (internal/process/task_queues.js:95:5)

Because Redis receives the following command:

"FT.SEARCH" "TrainsIndex" "12*" "INFIELDS" "2" "tags number" "HIGHLIGHT" "TAGS" "<b>" "</b>"

But it should be:

FT.SEARCH "TrainsIndex" "12*" INFIELDS 2 tags number HIGHLIGHT TAGS "<b>" "</b>"

I tried writing my own query with client.sendCommand but it resulted in the same query as what the library does, so I hope there is some kind of workaround because I couldn't find the implementation of the multi function in this library.

Add debugging logs

Add ability to enable debugging logs in order for better bug reporting + easier understanding of issues

Add custom Error handling to all modules

Today, we're returning the response of each command executed, that is useful while negative testing but in all other cases it's better throw an exception (With ability to control it, ofcourse) to stop code execution right away.

Solution:
Add an error handling system where, in one case returning the error as is and in other case to throw an exception.

[RediSearch] Add support for FT.CREATE & FT.SEARCH attributes

Two more features were added specific to JSON:

On FT.CREATE:

• identifier AS Attribute

On FT.SEARCH:

• RETURN {jsonPath} AS {property}

dependency update: ioredis

at the moment ioredis 4.17 (~1 year old) is used, the current version is 4.27

Thanks!

Add support for `ON JSON` index type in Search

Add an SDK playground

The most annoying thing, is to install/download something and after that understanding it sucks/doesn't fit you well.
We should have an SDK to play round with the module classes latest codes!

Refactor Redis All-in-one class

Aggregation slimmed down

Package limits how you can preform aggregations.

PR to expand #103

Broken links in README

All links of Quick start and Functions are broken because of the recently added icons

Support json.toggle and json.clear commands

Add support for the following commands:
json.toggle
json.clear

Fix SCANDUMP & LOADCHUNK tests for Redisbloom-CMK & Redisbloom

Redis parameters

Is there a way to pass parameters?
For example when I submit a query with a $variable is I get an error message about missing parameters.

Kind of like https://redisgraph.github.io/redisgraph.js/Graph.html#query

RedisBloom: Support for T-Digest data structure

Given RedisBloom/RedisBloom#285 there is the following set of commands we should enable:

• TDIGEST.CREATE: Allocate a new histogram
• TDIGEST.RESET: Empty out a histogram and re-initialize it
• TDIGEST.ADD: Add a value to the t-Digest with the specified count
• TDIGEST.MERGE: Merge one t-Digest into another
• TDIGEST.CDF: Returns the fraction of all points added which are ≤ x.
• TDIGEST.QUANTILE: Returns an estimate of the cutoff such that a specified fraction of the data added to the t-Digest would be less than or equal to the cutoff.
• TDIGEST.MIN: Get the minimum value from the histogram. Will return DBL_MAX if the histogram is empty
• TDIGEST.MAX: Get the maximum value from the histogram. Will return DBL_MIN if the histogram is empty
• TDIGEST.INFO : Returns compression, capacity, total merged and unmerged nodes, the total compressions
  made up to date on that key, and merged and unmerged weight.

With the in-depth params in https://oss.redislabs.com/redisbloom/master/TDigest_Commands/

Update RedisAI API

note that we are making changes to RedisAI commands
the following commands will be deprecated and you should use the new command:
MODELSET -> MODELSTORE https://oss.redislabs.com/redisai/master/commands/#aimodelstore
MODELRUN -> MODELEXECUTE https://oss.redislabs.com/redisai/master/commands/#aimodelexecute
SCRIPTRUN -> SCRIPTEXECUTE open for review - RedisAI/RedisAI#682
DAGRUN -> DAGEXECUTE

Adding Object based response for RediSearch

Today, RediSearch is returning complex responses that are not comfortable to handle/manage.

Solution:
Create a type/object that will hold this information

how to return just 2 or more fields

          return: {
            num: 1,
            field: 'tenant'
          }

is working but if I want more than one field:

          return: {
            num: 2,
            field: 'tenant _id'
          }

throws an error

TimeSeries: Add support for FILTER_BY_TSE/VALUE in TS.RANGE/TS.REVRANGE

TS.RANGE key fromTimestamp toTimestamp [FILTER_BY_TS TS1 TS2 ..] [FILTER_BY_VALUE min max] [COUNT count] [AGGREGATION aggregationType timeBucket]

TS.REVRANGE key fromTimestamp toTimestamp [FILTER_BY_TS TS1 TS2 ..] [FILTER_BY_VALUE min max] [COUNT count] [AGGREGATION aggregationType timeBucket]

add support for setting [NX | XX] on json.set

hey can you add support for this?
doc's: https://oss.redislabs.com/redisjson/commands/#syntax_3

Thanks

RediSearch HSET

Since ft.add is depreciated they say you should use HSET because RedisSearch 2.0 is indexing on HSET.
See https://oss.redislabs.com/redisearch/Commands/#ftadd

How do I have access to HSET using your module?

Support json.toggle and json.clear commands

Add support for the following commands:
json.toggle
json.clear

Add missing JSON function to the Redis all in one ft

Set data wipe policy

I have not configured the sdk to wipe data from the storage (TTL in redis), but the data that I store in the container (written via the sdk interface)) is wiped after regular intervals of time.

I am sure that this is not a problem with the container as the container has not crashed since launch.

Add linting

1. Add linting using ESlint and set TS recommended linting
2. Add linting in the CI Setup build

Add AI module class

The package is missing the AI module. we need to fill it in

Fix Redisearch tests

The following tests are failing:

RediSearch Module testing
alter function:
Error: Redisearch: ReplyError: Duplicate field in schema
at Redisearch.handleError (modules/module.base.ts:35:19)
at Redisearch. (modules/redisearch.ts:294:25)
at Generator.throw ()
at rejected (modules/redisearch.ts:6:65)
at processTicksAndRejections (internal/process/task_queues.js:93:5)

2. RediSearch Module testing
   tagvalgs function:
   Error: Redisearch: ReplyError: Not a tag field
   at Redisearch.handleError (modules/module.base.ts:35:19)
   at Redisearch. (modules/redisearch.ts:370:25)
   at Generator.throw ()
   at rejected (modules/redisearch.ts:6:65)
   at processTicksAndRejections (internal/process/task_queues.js:93:5)

Split shared content of modules into base module class

Create a 'base' module class for shared functionality between different module classes

Move Documentation to Project Wiki

TBD

Add RedisAI doc to the README

A new RedisAI class was added, should have docs as well

Adding docs for Redis all in one

Add config function to RedisGraph module

Unicode issue

When using

await this.client.set('api:cities', '.', '{"Mingá": {"id": "18", "name": "Mingá"}}');

in Redis I got this

{"Ming\u00c3\u00a1":{"id":"18","name":"Ming\u00c3\u00a1"}}

When using the redis-cli --raw I can handle keys with unicode, so I wondering how to make the set function to insert 'Mingá' as a key and not "Ming\u00c3\u00a1"??

Redisearch: Adding 2 fields with weight results in error

When create 2 fields and one with weight and sortabel ({name: 'blabla1', type: 'TEXT', sortable: true}) it results with I get the error: ReplyError: Invalid field type for field WEIGHT``

Aggregations ill formed object

Iv noticed that when running aggregation that there are at least 4 possible responses from the client.aggregate method

// returns single value in array as number - total value of objects
[0]

// expected results with a count and an array of values
[count, ...results]

// a object with the key as the total and as object of the result
{1: {a:'hello', b: 'world'}}

Then today i found a new one which i have no idea what to do with.

{
  "3": {
    "hour": "1629284400",
    "c": "bla",
    "A": "326450",
    "STARS": "666850",
    "RINGS": "495916",
    "ANSWERS": "0",
    "MASHUPS": "0",
    "RETIRES": "0",
    "PARSED": "0",
    "VOICEMAIL": "583",
    "ERRORS": "170934",
    "TIMESTAMP": "2021-08-18T11:00:00Z",
    "CARRIER": "bla"
  },
  "hour,1629280800,c,bla,A,576620,STARS,1244434,RINGS,934051,ANSWERS,0,MASHUPS,0,RETIRES,0,PARSED,0,VOICEMAIL,1025,ERRORS,310383,TIMESTAMP,2021-08-18T10:00:00Z,CARRIER,bla": {
    "hour": "1629277200",
    "c": "bla",
    "A": "323226",
    "STARS": "558757",
    "RINGS": "442940",
    "ANSWERS": "0",
    "MASHUPS": "0",
    "RETIRES": "0",
    "PARSED": "0",
    "VOICEMAIL": "416",
    "ERRORS": "115817",
    "TIMESTAMP": "2021-08-18T09:00:00Z",
    "CARRIER": "bla"
  }
}

^^ ment to be 3 objects but, the first one has been inserted into the count object, the second has been inserted into the key on the 3rd object.

if you loop to identify key and item you get something like this

key:  3
item:  0
=============
key:  hour,1629280800,c,bla,A,576620,STARS,1244434,RINGS,934051,ANSWERS,0,MASHUPS,0,RETIRES,0,PARSED,0,VOICEMAIL,1025,ERRORS,310383,TIMESTAMP,2021-08-18T10:00:00Z,CARRIER,bla
item:  1
=============

RediSearch - Add support for RETURN 0

Hello, In the doc's for RETURN parameters it says:
"If num is 0, it acts like NOCONTENT " - https://oss.redislabs.com/redisearch/Commands/#ftsearch

There doesn't seem any way to do this with this modules, return: { num: 0} doesn't work.

maybe you can add support for this ? Thanks :)

Please add the import statement to example page

How should we require or import for each module?

Running testcafe test in latest version (0.5.1) fails with 'Error: TypeScript compilation failed.'

running testcafe test, that uses import of ReJSON, Redisearch, RedisBloom, RedisGraph, RTS, GraphConfigInfo from 'redis-modules-sdk' fails with error:
Error: TypeScript compilation failed.
node_modules/redis-modules-sdk/lib/modules/redis.d.ts (46, 28): ']' expected.
node_modules/redis-modules-sdk/lib/modules/redis.d.ts (46, 42): ';' expected.
node_modules/redis-modules-sdk/lib/modules/redis.d.ts (46, 43): Declaration or statement expected.
node_modules/redis-modules-sdk/lib/modules/redis.d.ts (47, 1): Declaration or statement expected.

RTS: mrange and mrevrange should'nt expect a key

Today, both functions expect a key which incorrect, the key argument should be removed.
The code today results in the error ReplyError: ERR TSDB: wrong fromTimestamp uppon usage.

In the CLI, TS.MRANGE key:2:32 1606210883758 1606210893759 FILTER label!=x is expected to be TS.MRANGE 1606210883758 1606210893759 FILTER label!=x

Adding Redis Interval Sets module

Adding the https://github.com/danitseitlin/redis-interval-sets module

add Redis Cluster support

Hi there,

ioredis has support for Redis Cluster connections.
Currently I dont see a way to connect to Clusters with redis-modules-sdk.

may be all that has to be changed is:

/modules/module.base.ts line 24 from

    async connect(): Promise<void> {
        this.redis = new IORedis(this.redisOptions);
    }

to something like

    async connect(): Promise<void> {
        if (this.redisOptions.cluster) // flag tbd
           this.redis = new IORedis.Cluster(this.redisOptions);
        else
           this.redis = new IORedis(this.redisOptions);
    }

This should work with "default" Redis, I dont know if this will work with Redis modules and saidly cannot test it atm.

Add support for TS.DEL

TS.DEL

Delete data points for a given timeseries and interval range in the form of start and end delete timestamps.

The given timestamp interval is closed (inclusive), meaning start and end data points will also be deleted.

TS.DEL key fromTimestamp toTimestamp

• key - Key name for timeseries

• fromTimestamp - Start timestamp for the range deletion.
• toTimestamp - End timestamp for the range deletion.

Complexity

TS.DEL complexity is O(n).

n = Number of data points that are in the requested range

Delete range of data points example

TS.DEL temperature:2:32 1548149180000 1548149183000

Add contributing doc

In order for easier contributing there is a need to add a Contribution doc

TimeSeries: Add aggregation ALIGN option for TS.RANGE/REVRANGE and TS.MRANGE/MREVRANGE

RedisTimeSeries/RedisTimeSeries#801

Detailed TS.RANGE args with the new ALIGN feature

TS.RANGE key fromTimestamp toTimestamp [FILTER_BY_TS TS1 TS2 ..] [FILTER_BY_VALUE min max] [COUNT count] [ALIGN value] [AGGREGATION aggregationType timeBucket]
TS.REVRANGE key fromTimestamp toTimestamp [FILTER_BY_TS TS1 TS2 ..] [FILTER_BY_VALUE min max] [COUNT count] [ALIGN value] [AGGREGATION aggregationType timeBucket]

Detail of ALIGN docs

• ALIGN - Time bucket alignment control for AGGREGATION. This will control the time bucket timestamps by changing the reference timestamp on which a bucket is defined.
  Possible values:

  • start or -: The reference timestamp will be the query start interval time (fromTimestamp).
  • end or +: The reference timestamp will be the signed remainder of query end interval time by the AGGREGATION time bucket (toTimestamp % timeBucket).
  • A specific timestamp: align the reference timestamp to a specific time.

  Note: when not provided alignment is set to 0.

---

Sample behaviour:

( first ingestion )

127.0.0.1:6379> ts.add serie1 1 10.0
(integer) 1
127.0.0.1:6379> ts.add serie1 3 5.0
(integer) 3
127.0.0.1:6379> ts.add serie1 11 10.0
(integer) 11
127.0.0.1:6379> ts.add serie1 21 11.0
(integer) 21

Old behaviour and the default behaviour when no ALIGN is specified ( aligned to 0 ):

127.0.0.1:6379> ts.range serie1 1 30 AGGREGATION COUNT 10
1) 1) (integer) 0
   2) 2
2) 1) (integer) 10
   2) 1
3) 1) (integer) 20
   2) 1

Align to the query start interval time (fromTimestamp)

127.0.0.1:6379> ts.range serie1 1 30 ALIGN start AGGREGATION COUNT 10
1) 1) (integer) 1
   2) 2
2) 1) (integer) 11
   2) 1
3) 1) (integer) 21
   2) 1

Align to the query end interval time (toTimestamp). The reference timestamp will be the signed remainder of query end interval time by the AGGREGATION time bucket (toTimestamp % timeBucket).

127.0.0.1:6379> ts.range serie1 1 30 ALIGN end AGGREGATION COUNT 10
1) 1) (integer) 0
   2) 2
2) 1) (integer) 10
   2) 1
3) 1) (integer) 20
   2) 1

Align to a timestamp

127.0.0.1:6379> ts.range serie1 1 30 ALIGN 1 AGGREGATION COUNT 10
1) 1) (integer) 1
   2) 2
2) 1) (integer) 11
   2) 1
3) 1) (integer) 21
   2) 1