bajankristof/nedb-promises

A dead-simple promise wrapper for nedb.

A dead-simple promise wrapper for nedb.

Check out the docs.

IMPORTANT

As of nedb-promises 5.0.0 nedb package has been replaced with a fork of the original package, @seald-io/nedb to solve some vulnerability issues originating from nedb!

const Datastore = require('nedb-promises')
let datastore = Datastore.create('/path/to/db.db')

// #1
datastore.find({ field: true })
  .then(...)
  .catch(...)
  
// #2
datastore.find({ field: true })
  .exec(...)
  .then(...)
  .catch(...)

// #1 and #2 are equivalent

datastore.findOne({ field: true })
  .then(...)
  .catch(...)
  
datastore.insert({ doc: 'yourdoc' })
  .then(...)
  .catch(...)
  
// or in an async function
async function findSorted(page, perPage = 10) {
  return await datastore.find(...)
      .sort(...)
        .limit(perPage)
        .skip(page * perPage)
}

Installation

npm install --save nedb-promises

Usage

Everything works as the original module, with a couple of exceptions:

• There are no callbacks.
• loadDatabase has been renamed to load.
• The cursor's projection method has been renamed to project.
• You should call Datastore.create(...) instead of new Datastore(...). This way you can access the original nedb properties, such as datastore.persistence.
• As of v2.0.0 the module supports events 😎... Check out the docs about events!

Check out the original docs!

load( )

You don't need to call this as the module will automatically detect if the datastore has been loaded or not upon calling any other method.

const Datastore = require('nedb-promises')
let datastore = Datastore.create('/path/to/db.db')
datastore.load(...)
  .then(...)
  .catch(...)

find( [query], [projection] ), findOne( [query], [projection] ), count( [query] )

These methods will return a Cursor object that works the same way it did before except when you call "exec" it takes no arguments and returns a Promise. The cool thing about this implementation of the Cursor is that it behaves like a Promise. Meaning that you can await it and you can call .then() on it.

const Datastore = require('nedb-promises')
let datastore = Datastore.create('/path/to/db.db')

//outside Promise chain
datastore.find(...)
  .then(...)
  .catch(...)
  
//insinde Promise chain
datastore.insert(...)
  .then(() => {
    return datastore.find(...)
  })
  .then(
    // use the retrieved documents
  )

;(async () => {
  await datastore.find(...).sort(...).limit()
})()

other( ... )

All the other methods will take the same arguments as they did before (except the callback) and will return a Promise.

Check out the docs.

nedb-promises's Issues

Remove value from sub array stored data

I've an array stored inside my datastore that is provided by an api service.
To access it, I need to use this code:

db.find({}).then( (result) => {
  console.log(result[0].myfield); 
});

the console log will return an array that will have this structure:

myfield: Array(10) // this is the value of the returned result set and has the index key 0  
0: {city: "Istanbul", cookies: true, ip: "185.87.123.74", protocol: "http", ipPort: "185.87.123.74:3838", …}
1: {port: "32108", cookies: true, protocol: "http", anonymity: "elite", ipPort: "103.251.176.22:32108", …}
2: {anonymity: "elite", https: true, referrer: true, userAgent: true, city: "Makhachkala", …}

If I want to remove a value from the sub array using it's key how I can do it? for example if I want to remove the key 2 that is inside myfield, what is the correct way to proceed?

Add a CHANGELOG or release notes

I see that you've recently bumped the major version to 4.0, but without a CHANGELOG or any release notes, there is no way to know what breaking changes you've introduced.

high severity vulnerability

Hello,

nedb dependency ( https://github.com/bajankristof/nedb-promises/blob/master/package.json#L26 ) has an high severity vulnerability due to a dependency on underscore

Since nedb is a dead project, a possible solution could be to rely on this forked and fixed version of the dependency: louischatriot/nedb#675 (comment) => https://www.npmjs.com/package/@yetzt/nedb

Add LICENSE file

I can see that the npm package is listed with ICS license, but still adding LICENSE file is super important because:

without a license, the default copyright laws apply, meaning that you retain all rights to your source code and no one may reproduce, distribute, or create derivative works from your work.

Also, this would give us opportunity to include the proper copyright message and attribution when using the package as third-party dependency.

support nedb package as plugin

Sure the orginal nedb is stable but it is not likely to ever be patched in any way.

Lots of folks have forked it and done various patches. This module as it stands can only use the original. It would be great if the nedb used was a configurable plugin with the original as the default.

Of course whatever nedb "flavor" one used it would need to honor the original api that the wrapper requires.

I have already forked this repo and am using an alternative nedb. It would be nice however to not maintain a fork for just this.

Plugins are pretty easy with dynamic imports which is now supported native in nodejs. It might be an excuse to update the code to esm. (would be a major version bump).

So if you are interested I'd do that as a PR

BTW as an extension to your class I have also added rxjs in which gives folks an additional option for listening for changes to a collection/document even deep within a single property.

Does this support nedb-core?

Nedb is dead or losing traction. A new project nedb-core has taken off in its place. Does this library aim to support nedb-core? Great library by the way. Saved me a ton of headache.

.

.

findOneOrCreate?

Are you looking for contributors? I have a few enhancements that I thought might benefit the project. Here is one:

  async findOneOrCreate(
    query: any,
    insert?: Partial<T>,
    projection?: Projection<T>
  ): Promise<T> {
    const rec = await this.findOne(query, projection)
    if (rec) return rec
    try {
      const newRec = await this.insert(insert || query)
      newRec.isNew = true
      return newRec
    } catch (e) {
      if (e?.errorType === 'uniqueViolated') {
        // concurrency protection
        // unique constraint violation, maybe it got inserted already
        // if it did, repeating this call should work without error
        return this.findOneOrCreate(query, insert, projection)
      }
      throw e
    }
  }

The motivation here is to solve for concurrency issues when using nedb in a server environment. This way multiple requests to create a record will gracefully return the expected result even when called concurrently.

Datastore not generic

Hi, I noticed that Datastore is not generic while in Nedb it is. Having it generic can help devs to only pass in correct data to e.g. insert calls.

Example with Nedb:

interface MyType {
  prop: string;
}

const db: Datastore<MyType> = new Datastore();

// somewhere else:
db.insert({ }, (err, doc) => { /* ... */ });
//        ^^^ typescript complains that "{}" is not assignable to interface "MyType",
//            since it is missing the property "prop".

Am I missing something or is this is currently not possible to get this kind of help from compiler?

Support for @seald-io/[email protected] (and later)

The current version of nedb-promises is affected by CVE-2021-43138 .

the dependency to async was removed in version 3.0.0 of @seald-io/nedb .

Also @seald-io/nedb has a Promise-based API beginning from 3.0.0. Unfortunately this new API is not compatible with the implementation of nedb-promises, as it adds the Async suffix to all the method names.

From my point of view, a lot of code of this repository could be refactored to internally use the Async methods.

But more important would be to get rid of the security issue.

Package contains a reference to a license without a matching LICENSE file.

Your LICENCE file states, that nedb-promises is released under MIT License, but your package.json states, that it is released under ISC License.

As recommended by socket.dev:
A new version of the package should be published that includes a single license. Consumers may seek clarification from the package author. Ensure that the license details are consistent across the LICENSE file, package.json license field and license details mentioned in the README.

I would recommend to adjust, the package.json to

{
  // ...
  "license" : "MIT",
  // ...
}

as you reference the MIT License in the LICENSE file

Implement "compaction.done" event from persistence

Hello
can you implement the event compaction.done.

Proposal:

load() {
  if ( ! (this.__loaded instanceof Promise)) {
    this.__loaded = new Promise((resolve, reject) => {
      this.__original.loadDatabase((error) => {
        if (error) {
          this.emit('loadError', this, error)
          this.emit('__error__', this, 'load', error)
          reject(error)
        } else {
          this.emit('load', this)
          resolve()
        }
      })
    })
  }
 // NEW
  this.__original.on("compaction.done", () => {
    this.emit("compaction.done");
  });
 // NEW END
  return this.__loaded
}

Return value of `find().exec()` is `T[][]`, should be `T[]`

The DataStore.find() method returns an object of type Nedb.Cursor<(T & Document)[]>, but Cursor<T>.exec() returns T[].

As a result, calling DataStore.find<T>(query).exec() returns type (T & Document)[][], when it should actually be returning (T & Document)[].

Count by timestamp

I want to find count with default timestamp 'createdAt'

At model

count(day) {
return this.db.count({ createdAt: { $gt: day, $lt: Date.now() } }).exec();
}

day timestamp

const lastDay = moment().subtract(7, 'days').valueOf();

function call

new Promise<void>((resolve, rejects) => {
maungitemstore.count(lastDay).then((result) => {
console.log(result);
return resolve();
});
});

But return value is 0. I already inserted five document

Q: Can I use camo with nedb-promises?

camo - https://github.com/scottwrobinson/camo

can I use it with this package, I am mainly interested in the type/validation it provides.

thanks

The first find function takes time

The first find function takes time.
But with the original NeDB, it's fast.

const Datastore = require('nedb-promises');
let datastore = Datastore.create('db.db');

async function dbTest()
{
// Test Data
let arry=[];
for(let i = 0; i < 1000; i++){
    if(i % 2 === 1) arry.push({color: 'blue'});
    else arry.push({color: 'red'});
}
datastore.insert(arry);

 let data1 = await datastore.find({ color: "red" }); //  ★take time　443.943ms Why?　
 let data2 = await datastore.find({ color: "red" }); // 28.405ms
}

Orignal

const Datastore = require('nedb')
let db = new Datastore({ filename: 'db.db' });
db.loadDatabase(function(err){
  db.find({ color: 'red' }, function (err, docs) {  
     // 16.761ms
  });
  db.find({ color: 'red' }, function (err, docs) {
    // 34.478ms
  });  
});

Datastore.persistence is undefined.

Datastore.persistence is undefined, therefore can't do NeDB documented functions like Datastore.persistence.compactDatafile();

Temporary workaround Datastore.__original.persistence.compactDatafile(); etc.

Reported this problem to both "nedb-promise" and "nedb-promises" projects. Similar project, similar problem :)

db.find({ planet: /ar/ } not work

Hi ,when I use find using a regular expression ,is does work ,for the code as follows, it should return Mars and Earth ,but actually return nothing ,is there any problem?

• not work

// Finding all planets whose name contain the substring 'ar' using a regular expression
db.find({ planet: /ar/ }, function (err, docs) {
  // docs contains Mars and Earth
});

• work

// Finding all planets in the solar system
db.find({ system: 'solar' }, function (err, docs) {
  // docs is an array containing documents Mars, Earth, Jupiter
  // If no document is found, docs is equal to []
});

Calling Datastore#persistence#compactDatafile is deprecated

I moved over to using nedb-promises over plain nedb due to their Typescript types having issues. So far, it's been working great. The only thing I've noticed is that calling

database.persistance.compactDatafile()

will print a warning to the console with the following message:

(node:13872) DeprecationWarning: @seald-io/nedb: calling Datastore#persistence#compactDatafile is deprecated, please use Datastore#compactDatafile, it will be removed in the next major version.

This is due to @seald/nedb having marked calls going via .persistence as deprecated. Instead, they want us to call the functions straight on the database object. Like this:

database.compactDatafile()

The nedb-promises types file currently does not expose these functions. But there is a work-around in the mean time, to cast the object to any before making the call. So the method is there, it just isn't exposed.

(database as any).compactDatafile();

TypeScript (plus suggestions and auto import in VS Code) fail due to inaccurate TypeScript declaration

index.d.ts declares a default export, however nedb-promises does not have a default export.

When using import Datastore from 'nedb-promises', TypeScript will not throw an error when compiling, but the resulting JavaScript will throw an error when ran.

import Datastore from "nedb-promises"

const db = Datastore.create({filename: 'data.db'})

const db = Datastore.create({filename: 'data.db'})
                     ^

TypeError: Cannot read properties of undefined (reading 'create')

Note that this will not throw when tsconfig.json contains esModuleInterop: true, however this is not the default.

---

This also causes VS Code to incorrectly import nedb-promises in TypeScript and JavaScript projects.

Since index.d.ts tells VS Code that nedb-promises has a default export, VS Code suggests members that don't actually exist (because nedbPromises.default does not exist).

const db = nedbPromises.create({filename: 'data.db'})
                        ^

TypeError: Cannot read properties of undefined (reading 'create')

When the correct require statement is used, VS Code displays incorrect suggestions.

---

The solution to this is to use the export = syntax in index.d.ts. This will inform TypeScript that nedb-promises' export is not an ECMAScript Modules default export but a CommonJS export. TypeScript users must then use the import = require() syntax to import nedb-promises, or set esModuleInterop: true in tsconfig.json if they want to use the import from syntax to import nedb-promises.

TypeScript will inform the user of this in an error message when compiling if they try to using import from without esModuleInterop: true, so this will catch the resulting runtime error before it can occur.

This will also cause VS Code to auto import nedb-promises correctly in TypeScript and JavaScript projects.

Add `updateOne` and `updateMany` over `update`

Hey,

I'm using nedb-promises for my tests, whereas live I'm using mongodb, recent versions of mongo have started throwing errors when using update:

(node:76674) [MONGODB DRIVER] Warning: collection.update is deprecated. Use updateOne, updateMany, or bulkWrite instead.
(Use node --trace-warnings ... to show where the warning was created)

Without over engineering some alias for in my codebase I wondered if this package should be brought up to date. I'm happy to contribute if there are contribution guidelines.

Thoughts?

How can I use this via the <script> tag?

Hi

How can I get a compiled dist for this that I can use in the script tag of my app?

Thanks

Uncaught TypeError: Unexpected object, expected: Cursor

When I try to use the find or findOne method in the Electron renderer process, I get the error.
It's ok in main process.

Update creates a new document with same ID instead of replacing old one.

> const Datastore = require("nedb-promises");
> db = new Datastore({ filename: path });
> db.load()
> db.insert({name: "Klaus", age: 12, msgs: []})
> db.insert({name: "Tina", age: 13, msgs: []})
{"name":"Klaus","age":12,"msgs":[],"_id":"tsLAT46b58ioadgd"}
{"name":"Tina","age":13,"msgs":[],"_id":"KFqE0KPY5Z1zY6To"}
> db.update({name: "Klaus"}, {$set:{name: "Bert"}}, {}).then(console.log("true")).catch(err => console.log(err));
true
{"name":"Klaus","age":12,"msgs":[],"_id":"tsLAT46b58ioadgd"}
{"name":"Tina","age":13,"msgs":[],"_id":"KFqE0KPY5Z1zY6To"}
{"name":"Bert","age":12,"msgs":[],"_id":"tsLAT46b58ioadgd"}

Am I missing something obvious here?
I found this solution louischatriot/nedb#530 but this method does not seem to exist in nedb-promises

> db.persistence.compactDatafile()
TypeError: Cannot read property 'compactDatafile' of undefined

Error: 100% of the data file is corrupt, more than given corruptAlertThreshold (10%)

Error: 100% of the data file is corrupt, more than given corruptAlertThreshold (10%). Cautiously refusing to start NeDB to prevent dataloss.
at LineStream. (E:\electron\Techorbit_desktop\v2\techrobit_desktop_v2\node_modules@seald-io\nedb\lib\persistence.js:278:27)
at LineStream.emit (node:events:390:28)
at endReadableNT (node:internal/streams/readable:1343:12)
at processTicksAndRejections (node:internal/process/task_queues:83:21) {
corruptionRate: 1,
corruptItems: 2,
dataLength: 2

when i use afterSerialization and beforeDeserialization it throws this error , without enryption it works well...please advice

typescript interfaces errors

The typescript definition for the methods seems to be incorrect for insert when inserting multiple documents.

store.insert<T[]>(docs) seems to have the return type Promise<T[] & Document> when it should actually be Promise<(T & Document)[]>

findOne crashes when called on Proxy created by create()

Sorry my first conclusion was not correct.
It turns out that the __original property is not resolved correctly when findOne is called on an instance created by the create() function.
When the instance is created using the new keyword everything works fine.

Type definitions for Cursor.exec incorrect

In index.d.ts:
exec(callback: (err: Error, documents: T[]) => void): void
should be
exec(): Promise<T[]>

Datastore.create(path) doesn't create a file in the directory where it's called but in the process.env.pwd directory.

I spent a few hours trying to use this module but no matter how I rewrote it, I didn't manage to have Datastore.create(path) create a file at path. The documents were inserted but they were held in memory. In the end, I went for new Datastore and it solved the issue at once. however, I wouldn't be able to use persistence if this were the only way.

The solution I thought up is not optimal. I would use fs to know if the file has been created already. If it has, I can safely use Datastore.create, but if not, I will need to use new Datastore, restart the app, then Datastore.create in order to use persistence.

webpack 5 compatibility

./node_modules/@seald-io/nedb/lib/persistence.js:3:13-28 - Error: Module not found: Error: Can't resolve 'path' in '/../node_modules/@seald-io/nedb/lib'

BREAKING CHANGE: webpack < 5 used to include polyfills for node.js core modules by default.
This is no longer the case. Verify if you need this module and configure a polyfill for it.

If you want to include a polyfill, you need to:
- add a fallback 'resolve.fallback: { "path": require.resolve("path-browserify") }'
- install 'path-browserify'
If you don't want to include a polyfill, you can use an empty module like this:
resolve.fallback: { "path": false }

Missing Installation Guide

Thanks for such a nice package. I am missing the line npm install nedb-promises --save in the README and Documentation page. Can you please add this line?

adding new entry on update

I am new to use nedb-promises and I am facing issue that , when I am updating any record that's inserting a new record with same id

Uncaught TypeError: 'set' on proxy: trap returned falsish for property '__loaded'

Hi , i am getting this error
on using Datastore.create('/path/to/db.db'); gives this error but new Datastore(...) works jst fine.

I:\Electron\udemy-dl-gui\node_modules\nedb-promises\lib\Datastore.js:30 Uncaught TypeError: 'set' on proxy: trap returned falsish for property '__loaded'
    at __original.loadDatabase (I:\Electron\udemy-dl-gui\node_modules\nedb-promises\lib\Datastore.js:30:19)
    at newArguments.(anonymous function) (I:\Electron\udemy-dl-gui\node_modules\nedb\lib\executor.js:29:17)
    at I:\Electron\udemy-dl-gui\node_modules\nedb\lib\persistence.js:308:15
    at I:\Electron\udemy-dl-gui\node_modules\nedb\lib\persistence.js:140:12
    at I:\Electron\udemy-dl-gui\node_modules\nedb\lib\storage.js:105:30
    at storage.flushToStorage (I:\Electron\udemy-dl-gui\node_modules\nedb\lib\storage.js:56:97)
    at I:\Electron\udemy-dl-gui\node_modules\nedb\node_modules\async\lib\async.js:597:23
    at fn (I:\Electron\udemy-dl-gui\node_modules\nedb\node_modules\async\lib\async.js:582:34)
    at Immediate.<anonymous> (I:\Electron\udemy-dl-gui\node_modules\nedb\node_modules\async\lib\async.js:498:34)
    at runCallback (timers.js:672:20)
    at tryOnImmediate (timers.js:645:5)
    at processImmediate [as _immediateCallback] (timers.js:617:5)

fineOne with sort

Hiya - I want to findOne, but say the oldest one so really I'd like to do something like 👍

await db.findOne({}).sort({createdAt:-1})

It seems this isn't possible ?

Types Definitions

I'm using this package in a Typescript project and I see that you don't have any type definition

Do you have plans to create them ?

Datastore file didnt get created

I have the problem that the database file didnt get created, but my data is saved somewhere.

I use Electron 14.0.0, Windows 10 and Angular 12.4.

When i use

this._datastoreDiary = Datastore.create(path + 'diary.db');

then the file doesnt get created. I already searched on my whole computer for that file, but it does not exist. Instead my data is saved somewhere else. If i restart my electron app my data still exists, but if i change the path, then all data is gone, but there is still no file.
Can someone explain me that?

[BUG] ensureIndex duplicating

The function ensureIndex is duplicating the code in the .db file but the docs say: " No need to remove any ensureIndex though, if it is called on a database that already has the index, nothing happens."

Before app inits I have the db.ensureIndex({ fieldName: 'name', unique: true }) and the line of code written in the .db file is being duplicated.

I've tried the same with '@seald-io/nedb' and its not being duplicated.

store images and files

Is possible to store images or files using this database?
I saw on the original repo issues that this database doesn't support blob so I'm not sure if I can use it to save images or files. I like the way nedb works so if I can avoid to use indexedDB to achieve my scope it will be great.

findOne gives type <T> but can be null

Problem:

Basically, the findOne call assumes something is found whether or not it actually was, and TypeScript therefore doesn't alert you of trying to get properties from null.

const data = await db.findOne<{ name: string }>({ id: 1 });
console.log(data.name); // No TypeScript warning, but throws a TypeError if no row has id = 1

The returned type becomes <T & Document> but I think it should be <(T & Document) | null>. Alternatively throw some error.

const data = await db.findOne<{ name: string }>({ id: 1 });
console.log(data.name); // TypeScript warning: Object is possibly 'null'
if (data) {
  console.log(data.name) // Safe
}

This may also apply to some other similar methods.

Side note: Giving it a type like findOne<T | null>() doesn't work either, it still becomes T & Document

Possible solution:

Change the return type for findOne from Promise<T & Document> to Promise<(T & Document) | null>. This provides the desired effect, but I haven't delved into possible side effects and may be considered a breaking change for users who don't do this type check in their code already (although they are at risk of TypeErrors if not)

How to extends Document ?

I create my own interface who extends Document, but i cant import { Document } from 'nedb-promises' ...
How i can do that ?

thx :1: