exweiv/weiv-data

Custom API Library for Wix sites to connect MongoDB. Designed to easily switch from wix-data APIs.

weiv-data's Issues

How to Create Issues?

1. Click New issue.
2. Find the related tag about your topic.
3. Explain the issue with any helpful detail including images, links or codes.
4. Submit new issue.

After you submit your issue anyone can reply to you watch out for updates and please reply to any extra question that will be asked to you to get more info about the issue/topic you've created.

Return IDs as String Always

Some functions returns _ids in ObjectID type. We need to convert any ObjectId into string based id so it can be used. (Only for main _id field)

Better Middleware Logic for All Functions

Currently we handle validations or other things (middlewares) manually by adding them just before the function start. We should create a more dynamic logic for middlewares and validators.

• Dynamic Middleware system that will be similar to express middlewares.
• Better validator functions which will work with new middleware system.

BUG, system no more creates the `WeivDataURIEncryptSecret` automatically.

Create Truncate Function

https://www.wix.com/velo/reference/wix-data/truncate

Better Error Handling

We should make error messages more clear and understandable by the users. Currently we have some general problems about errors in almost every single function of the library.

Managing Indexes with weiv-data APIs

Collection Level:

• Creating search index.
• Deleting an index.
• Checking if index exists.
• Listing search indexes.
• Updating a search index.
• Creating a normal index.
• Getting index information.
• Deleting a search index.
• Listing indexes.
  etc.

consistentRead <boolean> change to <string>

We can rename consistentRead as readPreference and change value type to string to let library allow mongodb readPreference values.

We can also add useful and important options or all of them in data options. So users can use mongodb based options directly from weiv-data.

Create Sync Plugins for WeivData and Wix Apps

Wix app collections should be able to sync to the MongoDB collections easily, this can be done by exporting functions into backend files like data.js and events.js.

.ne or similar filters is not able to use `null`

JPP Issue new enrollment

Create Update Function

Create Get Function

https://www.wix.com/velo/reference/wix-data/get

IDs returned from .include method with query is in ObjectId type instead of string.

.query or other functions that reacts like that should be fixed

Migrate to Another Documentation System

Currently we are using TypeDoc to automatically create the documentation based on JSDoc and .d.ts file/s. But in the future we are planning to switch to a better documentation system or create our own system.

When this happens we will also update some type safety features in our library to even provide better type safety with integrated TS types.

Convert _id fields into string when returning.

Any _id field that's in ObjectId type should be returned as string not in actual object version of it for easier use in frontend.

Custom Cache Rules for MongoClient (Connection)

Currently system uses 5 min of cache time for MongoClients that's saved to cache via NodeCache. We can enable users to set custom cache rules for connections (MongoClient).

This can add more control to developers for cold starts.

Test Functions

• insert
• update
• get
• save
• remove
• truncate
• insertReference
• replaceReferences
• isReferenced
• removeReferences

Security Checks and Middleware Validators

Even if the library following some general/basic rules about security or other things it's not good enough, we should add some important but basic security features to library internally so users (clients) don't need to add their own basic validators or security checks.

Clients can still add functions that will run before the call to prevent security issues, or validate data that they are sending to weiv-data.

Create Save Function

https://www.wix.com/velo/reference/wix-data/save

Return ObjectId for _id fields always.

In some cases weiv-data returns string based ids (string version of ObjectIds) it should return ObjectId based ids always.

Remove Single-Reference Fields

Instead of having single-reference fields we'll use multi-reference fields and if only one item needed the array will contain only one ObjectId. This will make development faster and easier and also faster operations in some cases.

Managing Collections with weiv-data APIs

• Creating collections with weiv-data
• Deleting collections with weiv-data
• Listing current collections in a database with weiv-data
• Renaming a collection with weiv-data

Note: in MongoDB inserting a document also creates a collection automatically.

Enable ID Type Selection Globally (String || ObjectId)

Currently it's not possible to use string based ids with this library (MongoDB supports string ids) because we are converting every single id to ObjectId even if it's string.

Usually it's better to use ObjectId especially in MongoDB but you may need different things with your ids.

We will add a feature that will let you choose between two id type

• ObjectId
• StringId

In this way we will convert the ids you sent or not based on the choosen option. And you'll be able to store string based ids like UUID.

When string is selected our library will use UUID to generate ids.

Create Dynamic Database + Collection Name

Instead of setting db name and collection name using two different strings. Use single string and split names with /.

Example:

const { dbName, collectionName } = spliceNames(dynamicName: string);

Updates to Query Result Class

• Create dynamic currentPage system.
• Check if hasNext and hasPrev works as expected.
• Test all features of query result.

Add a _id converter option and remove ObjectId type force

_id values can be any type, basically whatever it is. We will only convert _ids (when returning data) if convertIds option is true. We will modify helpers to ignore non ObjectId _id's and only convert ObjectId type _id's from string to ObjectId.

Optionally Encrypt and Decrypt Data

In Wix (wix-data) this is possible via the UI of the Wix CMS dashboard (you can't configure extra settings with APIs, but can enable/disable this feature from the UI) but in weiv-data library this is not possible (not provided by the library but can be developed by the users).

Users will be able to:

• Insert data (only selected fields) that's encrypted (encryption will be handled by the library).
• Read data in decrypted format (or in encrypted format) easily.
• Choose encryption method from the available list. (Will has default value, so it's optional)

In this way it will be possible to encrypt and decrypt data when reading or writing.

Extra info: MongoDB Atlas already encrypt data at rest. But it's always better to handle some part of the data (like passwords, passport numbers etc. PII) encryption manually when saving to disk other than MongoDB Atlas default encryption at rest.

Update Docs

Documentation should be updated including the README part with more clear setup flow and other details.

Missing function feature for `convertId`.

We don't have a feature to only return ObjectId from objectids and strings.

Query Class Testing

Check/test if query features are working as expected. Compare with wixData.query()

Create Insert Function

Create .insert() function.

Updates to Aggregate Result Class

• Create Single Options Object in constructor
• getTotalItems should use collection.countDocuments function of MongoDB (performance impact **)
• Create dynamic currentPage. (Currently query starts from 1 directly) Should calculate based on skip and total items

Some functions don't have try catch and BUGs in error handling

• add try catch for any function that's required
• make sure error details are shown correctly without any type errors/bugs

Connection URIs Storing Strategy Change

Instead of creating three different secret for each connection URI we can use only one and store each URI inside of a JS object. This may reduce cold starts in the library.

All URIs stored as JSON in secret manager and named as WeivDataURIs

We get all three URI from that JSON value.

Testing Features/Functions of 4.0.0

• aggregate function and it's general features
• query function and it's general features
• include feature of query function
• _version function
• convertId function
• createCollection
• deleteCollection
• renameCollection
• listCollections
• queryReferenced
• replaceReferences
• removeReference
• insertReference
• isReferenced
• insert
• update
• remove
• get
• native (to check if cursor is returned correctly)

After we have test all features and fixed BUGs (if we find any) and update the documentation with new details/examples we will publish v4.0.1 and weiv-data will have it's own skeleton ready version for the first time.

skeleton ready version means a version that has all the main and required/optional functionality without any important BUG or problem.

Create Remove Function

https://www.wix.com/velo/reference/wix-data/remove