tuplo/dynoexpr

Expression builder for AWS.DynamoDB.DocumentClient

dynoexpr's Issues

can't use comparison operator "<>" with boolean value

diff --git a/src/expressions/filter.test.ts b/src/expressions/filter.test.ts
index a6dff03..3d24117 100644
--- a/src/expressions/filter.test.ts
+++ b/src/expressions/filter.test.ts
@@ -10,7 +10,7 @@ describe('filter expression', () => {
       c: '>= 2',
       d: '< 3',
       e: '<= 4',
-      f: '<> 5',
+      f: '<> true',
       g: '> six',
       h: '>= seven',
       i: '< eight',
@@ -63,7 +63,7 @@ describe('filter expression', () => {
         ':v862c': 2,
         ':vbaf3': 3,
         ':v122c': 4,
-        ':v18d5': 5,
+        ':v18d5': true,
         ':vb2dc': 6,
         ':v2543': 7,
         ':v60bf': 'six',

this change breaks the unit test.

# filter.test.ts

Error: expect(received).toStrictEqual(expected) // deep equality

- Expected  - 2
+ Received  + 2

@@ -15,11 +15,10 @@
      "#ncce7": "f",
      "#nec32": "e",
    },
    "ExpressionAttributeValues": Object {
      ":v122c": 4,
-     ":v18d5": true,
      ":v2543": 7,
      ":v51f2": "bar",
      ":v5c24": "me",
      ":v60bf": "six",
      ":v7b79": "you",
@@ -29,9 +28,10 @@
      ":v9a54": "nine",
      ":va4d8": "foo",
      ":vaa5c": "eight",
      ":vb2dc": 6,
      ":vbaf3": 3,
+     ":vcb09": "true",
      ":vd432": "seven",
    },

dynoexpr doesn't support if_not_exists

Current behavior

Right now the output of the code below looks like this:

import dynoexpr from '@tuplo/dynoexpr';

const params = dynoexpr({
  Update: { number: 'if_not_exists(420)' },
});

/*
{
  params: {
    UpdateExpression: 'SET #n15df = :vc32f',
    ExpressionAttributeNames: { '#n15df': 'number' },
    ExpressionAttributeValues: { ':vc32f': 'if_not_exists(420)' }
  }
}
*/

Expected behavior

With the support of if_not_exists the output should look like this:

/*
{
  params: {
    UpdateExpression: 'SET #n15df = if_not_exists(#n15df, :vc32f)',
    ExpressionAttributeNames: { '#n15df': 'number' },
    ExpressionAttributeValues: { ':vc32f': '420' }
  }
}
*/

Docs

https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Expressions.UpdateExpressions.html#Expressions.UpdateExpressions.SET.PreventingAttributeOverwrites

Remove dependency on AWS-SDK

We're only using the AWS-SDK to create a DynamoDBSet. Let's try and emulate it and drop the dependency

UpdateAdd attributes don't match

The ExpressionAttributeValues in the README match for UpdateAdd, but not in the test or the code: https://github.com/tuplo/dynoexpr/blob/main/src/expressions/update.test.ts#L218-L240

See :v0a80 not matching :v686c

const params = dynoexpr({
  UpdateAdd: {
    Color: ['Orange', 'Purple'],
  },
});
/*
  {
    UpdateExpression: 'ADD #n7295 :v0a80',
    ExpressionAttributeNames: { '#n7295': 'Color' },
    ExpressionAttributeValues: {
      ':v686c': Set { wrapperName: 'Set', values: [Array], type: 'String' }
    }
  }
*/

I'll dig into the code, but I'm wondering if I'm missing something with how I'm using it.

Dynoexpr doesn't allow to escape dynamic keys in objects

Current behavior

When I try to generate a command for an Update/UpdateRemove/Condition with a key in an object which is a simple string it works just fine:

import dynoexpr from '@tuplo/dynoexpr';

const dynamicKey = "key";

const params = dynoexpr({
  Update: { [`object.${dynamicKey}.value`]: `object.${dynamicKey}.value + 1` },
  Condition: { [`object.${dynamicKey}.value`]: "> 2" }
});

/*
{
  params: {
    ConditionExpression: "(#nbb017076.#nefd6a199.#n10d6f4c5 > :vaeeabc63)",
    ExpressionAttributeNames: {
      "#nbb017076": "object",
      "#nefd6a199": "key",
      "#n10d6f4c5": "value",
    },
    ExpressionAttributeValues: {
      ":vaeeabc63": 2,
      ":vc823bd86": 1,
    },
    UpdateExpression:
      "SET #nbb017076.#nefd6a199.#n10d6f4c5 = #nbb017076.#nefd6a199.#n10d6f4c5 + :vc823bd86",
  }
}
*/

The problem appears when the dynamicKey has . or a - character (maybe also a few others, but these ones I spotted in my project). When we change the key in such a way the output looks like this:

import dynoexpr from '@tuplo/dynoexpr';

const dynamicKey = "key.with-chars";

const params = dynoexpr({
  Update: { [`object.${dynamicKey}.value`]: `object.${dynamicKey}.value + 1` },
  Condition: { [`object.${dynamicKey}.value`]: "> 2" }
});

/*
{
  params: {
    ConditionExpression:
      "(#nbb017076.#nefd6a199.#n8b145925.#n10d6f4c5 > :vaeeabc63)",
    ExpressionAttributeNames: {
      "#nbb017076": "object",
      "#nefd6a199": "key",
      "#n8b145925": "with-chars",
      "#n10d6f4c5": "value",
    },
    ExpressionAttributeValues: {
      ":vaeeabc63": 2,
      ":vc823bd86": 1,
    },
    UpdateExpression:
      "SET #nbb017076.#nefd6a199.#n8b145925.#n10d6f4c5 = :veb17932c - :veb17932c - :vc823bd86",
  },
}
*/

As you can see, the object's nesting is broken and the update expression is generated incorrectly (the + sign has changed to the - sign).

Expected behavior

There should be a possibility to deal with such keys so the output would look like this:

/*
{
  params: {
    ConditionExpression: "(#nbb017076.#nefd6a199.#n10d6f4c5 > :vaeeabc63)",
    ExpressionAttributeNames: {
      "#nbb017076": "object",
      "#nefd6a199": "key.with-chars",
      "#n10d6f4c5": "value",
    },
    ExpressionAttributeValues: {
      ":vaeeabc63": 2,
      ":vc823bd86": 1,
    },
    UpdateExpression:
      "SET #nbb017076.#nefd6a199.#n10d6f4c5 = #nbb017076.#nefd6a199.#n10d6f4c5 + :vc823bd86",
  }
}
*/

Potential fix

Maybe there should be a possibility to wrap such keys in some special characters so dynoexpr could know that this string is a single key, e.g.:

`object."${dynamicKey}".value`

Breaking Change Not Documented

When I upgraded to v3.x from v2.x, I had to change my import to get it to work. This should be documented in the release.

// Old
const dynoexpr = require('@tuplo/dynoexpr');

// New
const {default: dynoexpr} = require('@tuplo/dynoexpr');

parseInValue: Regex attempts to parse invalid Condition Expressions in Filter strings

Desc

---

• Experienced some issues when a params.Filter is defined as Inspector or any string that starts with In.
• The parseInValue helper is triggered by the Regexp here, but it is not parsed by the tighter Regexp in the actual parseInValue helper: /^in\s*\(([^)]+)/i. This throws an error (will try to nab a screenshot of it).

Main Question: Is there a reason that different Regex expressions are used in these different places? It would seem like the Regex used in parseInValue should be used on ln. 75 (and so on for all the other helpers), otherwise regular strings will be unintentionally parsed as condition expressions.

logical operator "OR" doesn't work

Here's the example code

`var dynoexpr = require("@tuplo/dynoexpr")

const itemParams = dynoexpr({
Update: {
modified: new Date().toJSON(),
GSI1_PK:"OPEN",
GSI2_PK:"REQUEST#STATUS#open#DATE#2022-03-01t13:58:09.242z"
},
Condition: {
status: ["IN_PROGRESS", "OPEN"],
},
logicalOperator: 'OR',
})
console.log(
itemParams,
)`

and here's the result

{ logicalOperator: 'OR', ConditionExpression: '(#n6258 = :vccc9) AND (#n6258 = :vf5c3)', ExpressionAttributeNames: { '#n6258': 'status', '#n98d5': 'modified', '#n2461': 'GSI1_PK', '#nb7bd': 'GSI2_SK' }, ExpressionAttributeValues: { ':vccc9': 'IN_PROGRESS', ':vf5c3': 'OPEN', ':vbe1d': '2022-03-01T19:54:02.925Z', ':v5c8a': 'REQUEST#STATUS#open#DATE#2022-03-01t13:58:09.242z' }, UpdateExpression: 'SET #n98d5 = :vbe1d, #n2461 = :vf5c3, #n6258 = :vf5c3, #nb7bd = :v5c8a' }

condition expression should be OR not AND according to documentation

High Severity Security Issue In aws-sdk Dependency

As per the title, the version of the aws-sdk used in the library is very old and there's been a reported vulnerability here:

GHSA-rrc9-gqf8-8rwg

wrap arrays in createSet when using ADD and DELETE update actions

@mikestopcontinues said:

does it make sense to automatically wrap any arrays in createSet() when using ADD and DELETE update actions? Does that get in the way of any other behavior?

Condition with just attribute_not_exists throws Dynamo error

Issue

---

When putItem or updateItem operation is run with the condition specified below, Dynamo generates an error: "message": "ExpressionAttributeValues must not be empty"

const params = dynoexpr({
    Item: {
      pk,
      sk,
      attr1,
      attr2
    },
    // Failing condition:
    Condition: {
      pk: 'attribute_not_exists',
    },
    // Correct conditional string:
    // ConditionExpression: 'attribute_not_exists(sk)',
  })

I believe special checks should be added to the condition helper so only ExpressionAttributeNames is generated when the only conditions specified are either attribute_not_exists or attribute_exists.

Current Output:

"ConditionExpression": "(attribute_not_exists(#ne9e9))",
  "ExpressionAttributeNames": {
    "#ne9e9": "sk"
  },
  "ExpressionAttributeValues": {}

Expected Output:

"ConditionExpression": "(attribute_not_exists(#ne9e9))",
  "ExpressionAttributeNames": {
    "#ne9e9": "sk"
  }

Cannot find module 'dynoexpr' or its corresponding type declarations.

When I ran the following command,

sh -ex <<'__EOF__'
mkdir dynoexpr-test
cd dynoexpr-test
npm init --yes
npm install --save-dev typescript
npm install @tuplo/dynoexpr
echo "import dynoexpr from '@tuplo/dynoexpr';" > index.ts
npx tsc --lib es2015 index.ts
__EOF__

I got the following error:

+ mkdir dynoexpr-test
+ cd dynoexpr-test
+ npm init --yes
Wrote to /tmp/dynoexpr-test/package.json:

{
  "name": "dynoexpr-test",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "keywords": [],
  "author": "",
  "license": "ISC"
}


+ npm install --save-dev typescript
npm notice created a lockfile as package-lock.json. You should commit this file.
npm WARN [email protected] No description
npm WARN [email protected] No repository field.

+ [email protected]
added 1 package from 1 contributor and audited 1 package in 0.974s
found 0 vulnerabilities

+ npm install @tuplo/dynoexpr
npm WARN [email protected] No description
npm WARN [email protected] No repository field.

+ @tuplo/[email protected]
added 1 package from 1 contributor and audited 2 packages in 1.194s
found 0 vulnerabilities

+ echo import dynoexpr from '@tuplo/dynoexpr';
+ npx tsc --lib es2015 index.ts
node_modules/@tuplo/dynoexpr/lib/index.d.ts:1:131 - error TS2307: Cannot find module 'dynoexpr' or its corresponding type declarations.

1 import { DynoexprInput, DynoexprOutput, BatchRequestInput, BatchRequestOutput, TransactRequestInput, TransactRequestOutput } from 'dynoexpr';
                                                                                                                                    ~~~~~~~~~~


Found 1 error.

After the above command, the following command was successful.

sh -ex <<'__EOF__'
cd dynoexpr-test
mkdir -p node_modules/@types/dynoexpr
curl -o node_modules/@types/dynoexpr/index.d.ts https://raw.githubusercontent.com/tuplo/dynoexpr/v1.11.0/src/%40types/dynoexpr.d.ts
npx tsc --lib es2015 index.ts
__EOF__

Please let me know if I make any mistakes.

Support for batch and transact expressions?

I really like that with basic methods, it's possible to simply pass the entire request object in. This doesn't work for batch and transaction requests, since the expressions are all nested. Is it on the roadmap to support parsing the complete expressions for batch and transaction requests?

Question: Support for multiple values for the same attribute

Hey everyone!

Currently there is support for having multiple conditions for the same attribute ?

I'm trying to build this condition:

{
   Condition: 'attribute_not_exists(key) OR key = :value'
}

Thank you

not contains not working

hi, I'm using the library just fine.

contains works fine, not contains doesn't.

example

contains >>>

const params = dynoexpr({
      TableName: 'tablename',
      IndexName: 'sk-indexname',
      KeyCondition: { sk: 'itemcode' },
      Filter: { status: 'contains(FAILED)' },
    });

// ---------------- result

query 
 {
  TableName: 'tablename',
  IndexName: 'sk-indexname',
  KeyConditionExpression: '(#ne9e9 = :v3713)',
  ExpressionAttributeNames: { '#ne9e9': 'sk', '#n6258': 'status' },
  ExpressionAttributeValues: { ':v3713': 'itemcode', ':v0419': 'FAILED' },
  FilterExpression: '(contains(#n6258,:v0419))'
}

not contains >>>

const params = dynoexpr({
   TableName: 'tablename',
   IndexName: 'sk-indexname',
   KeyCondition: { sk: 'itemcode' },
   Filter: { status: 'not contains(FAILED)' },
 });

/// --- result

query 
{
TableName: 'tablename',
IndexName: 'sk-indexname',
KeyConditionExpression: '(#ne9e9 = :v3713)',
ExpressionAttributeNames: { '#ne9e9': 'sk', '#n6258': 'status' },
ExpressionAttributeValues: { ':v3713': 'itemcode', ':vd663': 'not contains(FAILED)' },
FilterExpression: '(#n6258 = :vd663)' // something wrong
}

Please let me know if I make any mistakes.

Thank you!

"@tuplo/dynoexpr": "^2.4.0"

Array of objects

Hi,

Thanks for the awesome library.

I'm trying to update an item with a list of objects like so:

const params = dynoexpr({
    TableName: TABLE_NAME,
    Key: { uuid },
    Update: {
      RoomNumber: room.roomNumber,
      Bro: [{mate: 'bro'}],
    },
  });

However, since the array gets converted to a Set, and not a List, it doesn't support Map's inside of it. What is the recommended way for handling this situation?

A combination of UpdateRemove with Condition throws an error

Current behavior

When I use basic Update like below it works just fine:

import dynoexpr from '@tuplo/dynoexpr';

const params = dynoexpr({
  Update: { "parent.item": 1 },
  Condition: { "parent.item": "attribute_exists" },
});

/*
{
  params: {
    ConditionExpression: '(attribute_exists(#nb602.#n7ebc))',
    ExpressionAttributeNames: { '#nb602': 'parent', '#n7ebc': 'item' },
    UpdateExpression: 'SET #nb602.#n7ebc = :v849b',
    ExpressionAttributeValues: { ':v849b': 1 }
  }
}
*/

But when I use UpdateRemove it returns empty ExpressionAttributeValues object:

import dynoexpr from '@tuplo/dynoexpr';

const params = dynoexpr({
  UpdateRemove: { "parent.item": 1 },
  Condition: { "parent.item": "attribute_exists" },
});

/*
{
  params: {
    ConditionExpression: '(attribute_exists(#nb602.#n7ebc))',
    ExpressionAttributeNames: { '#nb602': 'parent', '#n7ebc': 'item' },
    UpdateExpression: 'REMOVE #nb602.#n7ebc',
    ExpressionAttributeValues: {}
  }
}
*/

And empty ExpressionAttributeValues triggers an error from DynamoDB:

ValidationException: ExpressionAttributeValues must not be empty

---

Side note: TypeScript right now forces me to pass a defined value to UpdateRemove object but in reality, I'm passing undefined - I want to remove this item anyway, it doesn't matter what value it has → UpdateRemove: { "parent.item": undefined }

Expected behavior

Empty ExpressionAttributeValues object is not returned like below:

import dynoexpr from '@tuplo/dynoexpr';

const params = dynoexpr({
  UpdateRemove: { "parent.item": 1 },
  Condition: { "parent.item": "attribute_exists" },
});

/*
{
  params: {
    ConditionExpression: '(attribute_exists(#nb602.#n7ebc))',
    ExpressionAttributeNames: { '#nb602': 'parent', '#n7ebc': 'item' },
    UpdateExpression: 'REMOVE #nb602.#n7ebc',
  }
}
*/

Type error on params being passed into `docClient.update`

Hi,

This is my code snippet:

export const updateOrderViewerItem = async ({ order }: Args) => {
  const docClient = new AWS.DynamoDB.DocumentClient();
  const { ORDER_VIEW_TABLE_NAME } = process.env;
  const dynamoOrder = mapOrderToDynamoFields(order);
  const { OrderUUID, CustomerId, ...restOrder } = dynamoOrder;

  // get original item, or do we use a patch?
  const params = dynoexpr({
    TableName: ORDER_VIEW_TABLE_NAME,
    Key: { OrderUUID, CustomerId },
    Update: {
      ...restOrder
    }
  });

  Log.debug('Params', params);

  await docClient.update(params).promise();

  return order;
};

I seem to be getting the following error:

Normally I would just cast the type like so:

const params = dynoexpr({
    TableName: ORDER_VIEW_TABLE_NAME,
    Key: { OrderUUID, CustomerId },
    Update: {
      ...restOrder
    }
  }) as UpdateItemInput;

However that produces this error:

I suspect TableName and Key being missing from the DynoexprOutput is the issue, as when I apply a type like so:

type PatchedDynoexprOutput = DynoexprOutput & { TableName: string; Key: any }

const params = dynoexpr({
  // truncated
  }) as PatchedDynoexprOutput;

Everything works fine. Happy to do a PR for this if this is a suitable solution.

FR: Support multiple update actions

UpdateExpression supports using all four actions across multiple properties simultaneously. It would be great to be able to generate those properties easily. Perhaps using {UpdateSet, UpdateRemove, UpdateAdd, UpdateDelete}? What do you think?

New build style in 1.24 and 1.25 causing new errors

When upgrading, we saw this:

Error [ERR_REQUIRE_ESM]: Must use import to load ES Module: /Users/.../node_modules/@tuplo/dynoexpr/dist/index.js
require() of ES modules is not supported.
require() of /Users/.../node_modules/@tuplo/dynoexpr/dist/index.js from /Users/.../src/clients/dynamo.ts is an ES module file as it is a .js file whose nearest parent package.json contains "type": "module" which defines all .js files in that package scope as ES modules.
Instead rename index.js to end in .cjs, change the requiring code to use import(), or remove "type": "module" from /Users/.../node_modules/@tuplo/dynoexpr/package.json.

Edge Case: Update attribute operator parsing issues

Issue

---

• In using this package, we have run into an issue where my update parameters are parsed out as math expressions, when they need to be interpreted as literal strings.

Example Case

In this case, the contract field should be interpreted as a literal string. Instead, it gets cut up into an equation and the resulting parameters result in an error (shown below).

const params = dynoexpr({
  Key: {
    pk: `project#${id}`,
    sk: `project#${id}#detail`,
  },
  Update: {
    contract: '10-20-001', // <- This gets spliced up as a math expression
    updated: Date.now(),
  },
  ConditionExpression: 'attribute_exists(sk)',
})

The params that caused the error:

Fix

• Suggestion is to add a flag to the update parameters which can tell getExpressionAttributes whether or not to parse operational values or interpret the attribute value literally, right here.

Feature Request: Support validation of sets

When creating sets via DynamoDB.DocumentClient.prototype.addSet directly, users are able to pass options in the shape of { validate: true } to perform client-side validation that the Set is homogenous before triggering the API call. It would be nice to pass-through this configuration option.

dynoexpr doesn't support list_append

Current behavior

Right now the output of the code below looks like this:

import dynoexpr from '@tuplo/dynoexpr';

const params = dynoexpr({
  Update: { numbersArray: 'list_append([1, 2])' },
});

/*
{
  params: {
    UpdateExpression: 'SET #n15df = :vc32f',
    ExpressionAttributeNames: { '#n15df': 'numbersArray' },
    ExpressionAttributeValues: { ':vc32f': 'list_append([1, 2])' }
  }
}
*/

Expected behavior

With the support of list_append the output should look like this:

/*
{
  params: {
    UpdateExpression: 'SET #n15df = list_append(#n15df, :vc32f)',
    ExpressionAttributeNames: { '#n15df': 'numbersArray' },
    ExpressionAttributeValues: { ':vc32f': '[1, 2]' }
  }
}
*/

Docs

https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Expressions.UpdateExpressions.html#Expressions.UpdateExpressions.SET.UpdatingListElements

`dynoexpr` should handle `null` value type

Hi! DynamoDB supports null as NULL. It would be great if the library/types matched this behavior.