Giter Site home page Giter Site logo

node-persist's Introduction

node-persist

(localStorage on the server)

Super-easy asynchronous persistent data structures in Node.js, modeled after HTML5 localStorage

Node-persist doesn't use a database. Instead, JSON documents are stored in the file system for persistence. Because there is no network overhead, node-persist is just about as fast as a database can get. Node-persist uses the HTML5 localStorage API, so it's easy to learn.

This is still a work in progress. Send pull requests please.

Note

  • This is not designed for large amounts of data, you can do way more than the 5MB limit imposed by the browsers but don't stretch it, in some cases you might need to load the whole storage into RAM, but normat getItem/setItem does not.
  • If you're looking for the version that supports both synchronous and asynchronous use [email protected]

Install

$ npm install node-persist

Basic Example

const storage = require('node-persist');

//you must first call storage.init or initSync
storage.initSync( /* options ... */ );
// or 
// storage.init( /* options ... */ );

// then anywhere else in your code
await storage.setItem('name','yourname')
console.log(await storage.getItem('name')); // yourname

Run the counter example:

$ cd examples/counter
$ node counter.js
$ open up localhost:8080

3.1.1 change logs

backward changes

  • Added the writeQueue* options, trying to resolve issue#108, see the API Documentation below.

3.0.0 change logs

Non-backward changes

  • All the *Sync functions were removed, every operation is now asynchronous
  • All the persist* functions were removed
  • Nothing is held up in RAM use your own memory caching module, i.e. nano-cache
  • Node 7.6+ is required now, we're using async/await
  • continuous and interval options were removed, since we immediately persist to disk now, asynchronously
  • forEach callback now accepts an object callback({key, value}) instead of 2 arguments callback(key, value)

2.0.0 change logs

Non-backward changes

  • filenames on the file system are now md5 hashed now and the structure of the saved data has changed to include the ttl in them.
  • no longer need/support a options.ttlDir, since the ttls are now stored in the same file as each value
  • added expiredInterval option
  • added forgiveParseErrors option

1.0.0 change logs

Mostly non-backward changes

  • storage.getItem() now returns a promise
  • storage.valuesWithKeyMatch() no longer accepts a callback
  • storage.values() no longer accepts a callback
  • storage.key() is gone
  • The default dir is now process.cwd() + (dir || '.node-persist/storage'), unless you use an absolute path
  • added storage.get(), alias to getItem()
  • added storage.set(), alias to setItem()
  • added storage.del(), storage.rm(), as aliases to removeItem()
  • Keys, on the file system are base64 encoded with the replacement of the /

API Documentation

async init(options, [callback])

if the storage dir is new, it will create it

Options

You can pass init() an options object to customize the behavior of node-persist

These are the defaults

await storage.init({
	dir: 'relative/path/to/persist',

	stringify: JSON.stringify,

	parse: JSON.parse,

	encoding: 'utf8',

	// can also be custom logging function
	logging: false,  

	// ttl* [NEW], can be true for 24h default or a number in MILLISECONDS or a valid Javascript Date object
	ttl: false,

	// every 2 minutes the process will clean-up the expired cache
	expiredInterval: 2 * 60 * 1000, 

    // in some cases, you (or some other service) might add non-valid storage files to your
    // storage dir, i.e. Google Drive, make this true if you'd like to ignore these files and not throw an error
    forgiveParseErrors: false,
	
	// instead of writing to file immediately, each "file" will have its own mini queue to avoid corrupted files, keep in mind that this would not properly work in multi-process setting.
	writeQueue: true, 
	
	// how often to check for pending writes, don't worry if you feel like 1s is a lot, it actually tries to process every time you setItem as well
	writeQueueIntervalMs: 1000, 
	
	// if you setItem() multiple times to the same key, only the last one would be set, BUT the others would still resolve with the results of the last one, if you turn this to false, each one will execute, but might slow down the writing process.
	writeQueueWriteOnlyLast: true, 
});

async getItem(key)

This function will get the value for that key stored on disk

let value = await storage.getItem('obj');

async setItem(key, value, [options])

This function sets 'key' in your database to 'value'

await storage.setItem('fibonacci',[0,1,1,2,3,5,8]);
await storage.setItem(42,'the answer to life, the universe, and everything.');
await storage.setItem(42,'the answer to life, the universe, and everything.', {ttl: 1000*60 /* 1 min */ });

* The only option available when calling setItem(key, value, option) is {ttl: Number|Date}

async updateItem(key, value, [options])

This function updates a 'key' in your database with a new 'value' without touching the ttl, however, if the key was not found or if it was expired a new item will get set

await storage.updateItem(42,'the answer to life, the universe, and everything.', {ttl: 1000*60*10 /* 10 minutes */ });
await storage.updateItem(42,'means nothing, do not trust wikipedia'); // ttl is still the same, will expired in 10 minutes since it was first set

* The only option available when calling updateItem(key, value, option) is {ttl: Number|Date}

async removeItem(key)

This function immediately deletes it from the file system asynchronously

await storage.removeItem('me');

async clear()

This function immediately deletes all files from the file system asynchronously.

await storage.clear();

async values()

This function returns all of the values

await storage.setItem("batman", {name: "Bruce Wayne"});
await storage.setItem("superman", {name: "Clark Kent"});
console.log(await storage.values()); //output: [{name: "Bruce Wayne"},{name: "Clark Kent"}]

async valuesWithKeyMatch(match)

This function returns all of the values matching a string or RegExp

await storage.setItem("batman", {name: "Bruce Wayne"});
await storage.setItem("superman", {name: "Clark Kent"});
await storage.setItem("hulk", {name: "Bruce Banner"});
console.log(await storage.valuesWithKeyMatch('man')); //output: [{name: "Bruce Wayne"},{name: "Clark Kent"}]
// also accepts a Regular Expression
console.log(await storage.valuesWithKeyMatch(/man/)); //output: [{name: "Bruce Wayne"},{name: "Clark Kent"}]

async keys()

this function returns an array of all the keys in the database.

console.log(await storage.keys()); // ['batman', 'superman']

async length()

This function returns the number of keys stored in the database.

console.log(await storage.length()); // 2

async forEach(callback)

This function iterates over each key/value pair and executes an asynchronous callback as well

storage.forEach(async function(datum) {
	// use datum.key and datum.value
});

Factory method

create(options) - synchronous, static method

If you choose to create multiple instances of storage, you can. Just avoid using the same dir for the storage location. You still have to call init after create - you can pass your configs to either create or init

const storage = require('node-persist');
const myStorage = storage.create({dir: 'myDir', ttl: 3000});
await myStorage.init();

Tests

npm install
npm test

node-persist's People

Contributors

akhoury avatar anthonywoodard avatar benmonro avatar broox avatar cas-c avatar cazzer avatar darthhater avatar davericher avatar davides avatar dependabot[bot] avatar eliran avatar gillesc avatar ionicabizau avatar jackhedaya avatar joepie91 avatar jtakalai avatar kingcody avatar kittygiraudel avatar kokeksibir avatar masonbedard avatar mathrobin avatar noahtkeller avatar sanghaklee avatar simonlast avatar spacepluk avatar sreject avatar toddwilson avatar vkamydo avatar ybcs avatar yurks avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

node-persist's Issues

Unexpected end of input error

I have come across a problem where a bug in my code causes one of the files let's call it 'batman' to be empty. The file is there but has nothing in it. When this happens node just does a dump. Is there a way to catch this error and just init 'batman' as empty?

Maybe just need to fix the storage.getItem method to return null if the file is hosed? Then I could just do this:

if(!storage.getItem('batman')){
storage.setItem('batman',[]);
}

You can duplicate this error by simply using your example to create the 'batman' file then edit it to delete everything in it. Let me know what you think.

will this scale well?

This looks great! But what about large amounts of records? Would it choke if I had 1 million, or 5 million? I was thinking to save the user's Json config file for a free learning site, and my user # might get up there.

Old version in NPM

The version currently available in NPM seems severely outdated (2012).
Could someone fix this please?

npm package repository

The npm package of 'node-persist' points to the following GitHub repository : https://github.com/neudesic/node-persist, which is not up to date and even contains errors.
As an example, the persistKeySync function contains the following call which is NOT synchronous :

fs.writeFile(path.join(options.dir, key), json,
    options.encoding, function (err) {
        if (err) throw err;
    }
);

Is it possible to change the repository pointed by the npm package so that it refers to your repository ?

can not get key, even after 10sec wait

Heya, it's me again. (I submitted an issue here once before)

I'm currently working on some school project and chose to use your module again. However I ran into some problems, I'll try to describe the problem as abstract as possible.

So basically what I'm doing is the following:

var storeage require('node-persist')

storage.initSync({/* Options */})

storage.setItem('foo', 'bar', function (err) {
  if (err) throw err
})

setTimeout(function () {
  storage.getItem('foo', function (err, val) {
    if (err) throw err
    console.log(val)
  })
}, 10000) // 10sec

This logs undefined

However if I run the following it totally works.

var storeage require('node-persist')

storage.initSync({/* Options */})

storage.setItem('foo', 'bar', function (err) {
  if (err) throw err

  storage.getItem('foo', function (err, val) {
    if (err) throw err
    console.log(val)
  })
})

I think it's weird that even after a 10sec wait the getItem function can't find the stored value...

my setup

As I mentioned above my setup doesn't exactly look like this, I build a wrapper around the storage module that checks if the value has already been stored first. But I believe this code would do the same. But if you believe this other code could potentially be what's breaking the code, I'll be hapy to share my source code with you!

Pull requests

A brief side-question: what kind of PRs are you interested in? I've written my own wrapper with some extra methods to provide some Redis-like functionality, but it's not clear to me whether this module is meant to be a minimal core, or whether that kind of functionality would be appreciated as a PR.

For reference, the following are the things I've implemented so far:

  • increment(key, amount = 1)
  • decrement(key, amount = 1)
  • addListItem(key, item)
  • removeListItem(key, item)
  • removeListItemByFilter(key, filterFunc)
  • setProperty(key, property, value)
  • removeProperty(key, property)

As my code is in Coffeescript and this project is not, it'd take some amount of effort and time to port it to 'vanilla' Javascript, so I figured I'd inquire about the demand for it first.

Promise returned by setItem is resolved only after persist

Documentation states that

in the case where you set options.interval or options.continuous=false, your (optional) callback or your returned promise from this function will still get resolved immediately, even if the value has not been persisted to disk yet

but the promise is actually resolved only after persist is executed, so my question is - is it a bug, or is the documentation wrong ?

directories-like-keys and illegal chars problem

If the key is the actual name of a folder it will give a error:

Uncaught Exception:
Error: ENOENT: no such file or directory, open
'persist/Users/herp/derp/gerp/nerp/burp'
at Error (native)

"process out of memory" error

I am getting the following error after initSync() runs for about 20 seconds:

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - process out of memory
Abort trap: 6

My storage folder contains ~1500 files with a total size of 750 MB.
nodes --max_old_space_size option doesn't seem to do anything.

As a workaround, is there any way to read objects one by one, instead of putting them all into memory at once?

Concurrency issues

When doing heavy concurrent writes/persists that progressively decrease in size, files can sometimes become corrupted as newer data only partially overwrites older data (as there is no locking mechanism in place).

I can write a testcase later if needed, as I'm currently rather busy, but I figured I'd file an issue before I would forget.

I did write a very hacky monkeypatch (in Coffeescript) that at least ensures write queueing:

writeQueue = []
currentlyWriting = false
_setItem = persist.setItem

persist.setItem = (key, value) ->
    new Promise (resolve, reject) ->
        _setItem.call(persist, key, value)
        addItemToQueue key, value, resolve, reject
        triggerWrite()

addItemToQueue = (key, value, resolveFunc, rejectFunc) ->
    writeQueue.push [key, value, resolveFunc, rejectFunc]

triggerWrite = ->
    if not currentlyWriting and writeQueue.length > 0
        currentlyWriting = 1
        [key, value, resolveFunc, rejectFunc] = writeQueue.shift()

        Promise.resolve(persist.persistKey(key))
            .then (result) -> resolveFunc(result)
            .catch (err) -> rejectFunc(err)
            .finally ->
                currentlyWriting = false
                triggerWrite()

It's very inefficient - it doesn't attempt to 'deduplicate' queued writes and it always persists the last known version at the time of persisting, so under heavy concurrency it'll likely end up writing identical data many times... but at least it doesn't corrupt the files on disk :)

EDIT: For clarification, these monkeypatched methods I wrote are what caused me to discover the issue:

persist.addListItem = (key, item) ->
    newList = [item].concat (persist.getItem(key) ? [])

    persist.setItem key, newList

persist.removeListItem = (key, item) ->
    newList = (persist.getItem(key) ? [])
        .filter (existingItem) ->
            return (item == existingItem)

    persist.setItem key, newList

persist.removeListItemByFilter = (key, filter) ->
    newList = (persist.getItem(key) ? [])
        .filter (item) ->
            return !filter(item)

    persist.setItem key, newList

Rapidly calling removeListItem several times (eg. like would happen in a queue) would result in the aforementioned writes that progressively decrease in size.

Unable to use data outside of promise?

Hi, I'm creating a wrapper module around yours to make it easier to use for my purposes.
However I'm having trouble while getting an item from storage.

Here's my code.

write: function (key, data, callback) {
    var storageObject = {
      // Add the data we want to store to the storage Object.
      data: data,
      // Add the time of creation to the storage Object.
      created: new Date().getTime()
    }

    /* Save the item to our node-persist cache. */
    storage.setItem(key, storageObject, callback);

  },
  read: function (key) {
    storage.getItem(key)
      .then(function (value) {
        console.log('promise: '+value.data);
        return value.data;
      });
  },

Those functions are both on module.exports

Then somewhere else in the file I do this:

module.exports.write('user', 'xxxx', function (err) {
  if (err) {
    console.log('ERROR');
  }
  console.log('user: '+module.exports.read('user'))
});

When I run this file in node I get the following
screen shot 2016-11-29 at 20 29 45

My question

What am I doing wrong here? I do return the value of the Object I'm getting at the same time of logging it, however It's undefined when logging it? This is weird in my opinion.

Relative paths are relative to wrong directory

When using the standard storage.initSync(), files are saved to ./persist, as expected. When using the factory method (sample code below), files are saved to ./node_modules/node-persist/src/storage.

Sample code:

var storage = require('node-persist');
var usersStorage = storage.create({dir: 'users'});
usersStorage.initSync();

var batman = {
    first: 'Bruce',
    last: 'Wayne',
    alias: 'Batman'
};

usersStorage.setItem('batman',batman);

In this example, the file 'batman' is saved to ./node_modules/node-persist/src/storage/users. The desired outcome is to have the file saved to ./persist/users.

Is this a bug or am I doing something wrong in my code?

Thanks in advance!

getItem returns empty string

I have a file .node-persist/storage/023a024fab1608b372fdef427ec098ea with regular json data on it. The problem is that the file does exist and all, but node-persist throws the following on the getItem:

/home/me/my-project/node_modules/node-persist/src/local-storage.js:578
            self.data[input.key] = input;
                           ^

TypeError: Cannot read property 'key' of undefined
    at /home/me/my-project/node_modules/node-persist/src/local-storage.js:578:28
    at tryToString (fs.js:455:3)
    at FSReqWrap.readFileAfterClose [as oncomplete] (fs.js:442:12)

Process finished with exit code 1

and I proved that is readable by executing this on node:

fs.readFile('.node-persist/storage/023a024fab1608b372fdef427ec098ea', 'utf8', (e, t) => { console.log(JSON.parse(t));});

returns a valid json printed on the terminal.

Doing some debugging on local-storage.js. Found out that the text param on the callback is an empty string(""), and that's why it can't be parsed and used as expected.

Any clues? Thanks

Can I persist functions or just non prototyped data

I'm searching for a simple way to persist a array between a master and child process. I want to know if with node-persist I can do that trick, withouy using eval or new Function(...) alternativies.

Thanks

Weird parsing errors? [google drive sucks]

Alright so, I use two main system to write all my code on. Yesterday, as you may remember I was working with your module as described in #71. Then today I came back to continue working on it and something strange started happening.

image

So I started commenting out the code that I thought might cause this weird error up until the point where I had only the following left...

 var storage = require('node-persist');

 /* Init our storage */
 storage.init({
   dir: './cachefolder'
 });

I then uncommented everything and saw that everything inside the file actually ran. This means that data was saved to the cache, it was then read from the cache and printed to the console. The data was retrieved from the cache once again and some stuff was calculated with the data upon which something else was printed.

Only after all this was done the error printed.

Also note that removing the part where I init the storage makes sure this error isn't thrown, but this is to be expected since the error occurs in one of the files inside of the node-persist storage module.

Redis adapter

Hi, First of all great and very simple project to use. Have you ever considered Redis for this as the optional storage (instead of filesystem) ?

Problem with installing

On windows, I have installed the package with

C:\dev\nodejs>npm install -g  node-persist

This seems to succeed. However, when I try to use it in the code

var storage = require('node-persist');

I get the following error message

C:\dev\eclipse64_kepler_wtp_workspace\P1_persistence>node hello_world_http.js

module.js:340
    throw err;
          ^
Error: Cannot find module 'node-persist'
    at Function.Module._resolveFilename (module.js:338:15)
    at Function.Module._load (module.js:280:25)
    at Module.require (module.js:364:17)
    at require (module.js:380:17)
    at Object.<anonymous> (C:\dev\eclipse64_kepler_wtp_workspace\P1_persistence\
hello_world_http.js:2:15)
    at Module._compile (module.js:456:26)
    at Object.Module._extensions..js (module.js:474:10)
    at Module.load (module.js:356:32)
    at Function.Module._load (module.js:312:12)
    at Function.Module.runMain (module.js:497:10)

Refactoring to use node-persist with async/await/Promise features of ES7

I hope this is not off-topic, I'd massively appreciate advice on this, don't know where else I could ask.

I was hoping for some advice on refactoring code to get the most out of ES7. I have looked at many examples of async/await/Promise features, and think I understand them, yet I cannot wrap my head around how to transform my code. Extracts are as follows:

I have a (simplified) wrapper module for node-persist that contains:

var storage = require('node-persist');
var mydb = 
{
    isReady: false,
    init: function()
    {
        storage.init().then(function(){
            mydb.isReady = true;
        });
    },
    store: function(key, val, onDone)
    {
        if (!mydb.isReady){
            console.error("Store: storage is not up yet. (Tried to store '"+key+"':'"+val+"')");
        } else {
            storage.setItem(key, val).then(onDone);
        }
    },
    retrieve: function(key, onDone)
    {
        if (!mydb.isReady){
            console.error("Retrieve: storage is not up yet. (Tried to retrieve '"+key+"')");
        } else {
            storage.getItem(key).then(onDone);
        }
    },
    //...
}
mydb.init();
module.exports = mydb;

Then, for example, I use the above code to modify a key/value pair like so:

function incrementFoo(onDone){
    mydb.retrieve("myObjectKey", function(myObject){
        myObject.foo++;
        mydb.store("myObjectKey", myObject, function(){
            onDone();
        })
    });
}

It all quickly leads to the Pyramid of doom in the rest of my code as the above function shows - I'd be extremely grateful for any help!

valuesWithKeyMatch as documented

This is from the documentation:

storage.setItem("batman", {name: "Bruce Wayne"});
console.log(storage.valuesWithKeyMatch('man')); //output: [{name: "Bruce Wayne"}

but local-storage.js implementation has
return match.indexOf(key) !== -1;
where match is "man" and key is "batman". It will not match because there is no "batman" in "man" but the other way around. I'm not sure if this is intentional and I'm just holding it wrong.

At any rate, I could fix it by wrapping my search string into RegExp.

Create tags on github

Hi,

I found your repository and it seems to do a great job with a nice simple API. I saw in the package.json that your current version is 0.0.11. I think it would be great to add gihub tag according to your releases in order to let people use an older version and do upgrade willingly and not each time we run npm install

Thank You,

doesn't work promise of init method

doesn't handle onSuccess callback, also doesn't log none of them from the snippet of code

storage.init({logging: true}).then(function() {
    console.log("loaded");
}, function(err) {
    console.log("error");
});

Can I contribute?

Very handy little utility. I'd like to add some features to this and convert it to an NPM module. If you give me access, I can do it directly on this repo or if you want I can do it on a fork. let me know...

tests

Im thinking mocha

File path issue.

Seems like there is some relative file path issue

Where node index.js console.logs getItem('foo') normally.

but node foo/index.js does not.

Program hangs using v2.0.4 and greater

I tried running this simple example using both 2.0.3 and 2.0.4. Here's the script:

var storage = require('node-persist');

storage.initSync();
storage.setItemSync('test', 'test')

Using 2.0.3, the script runs and I'm returned to the command prompt. This is expected.

Using 2.0.4, the program hangs and I have to use ctrl+c to end the process. Any idea why this is happening?

TTL not working....

Heya, I'm having some issues with the TTL option of your module and was wondering if you could help me out.

I have a pretty basic setup of your module and basically have a function that checks if a value has been added to the storage yet. If this is the case it returns the key at which it's stored. If not it creates a new key and adds the value to the storage then returns the newly created key to the callback function.

For this functionality I use the storage.forEach() method.

For testing purposes I set the TTL option in the initSync() function to 100. According to your documentation this should set the TTL to 100 milliseconds.

However when I run my testing function, a function that tries to add the same value to the storage every second, it returns the same KEY every single time.

Then there's this quote from your docs that gets me even more confused:

With ttl (time to live), it is recommended that you use getItem(key, callback) or getItemSync(key) since, if a ttl of a certain key is expired the key-file is immediately deleted from disk, the callback will execute whenever that happends, if there is no ttl used or it has expired yet, the callback will also immediately execute in a synchronous fashion.

I really hope you'll be able to help me get the TTL working. Thanks in advance!

My code

Down here I'll share my source code, this might make it easier for you to see what's going on.

This is my module:

var KEY = require('my-key-generation')

module.exports = function (PATH, TTL) {
  /* Initialize the storage */
  storage.initSync({
    dir: PATH,
    ttl: TTL
  })

  /* Add existing KEYs to our KEY generator: This way it knows what keys to skip.
   * NOTE: this function is sync, but that's okey. It'll only be called on init. */
  KEY.list = storage.keys()

  return {
    getKey: function (ID, callback) {
      var returnval, exists

      /* First check if the id is already stored. */
      storage.forEach(function (key, value) {
        if (value === ID) {
          /* We've already stored this ID */
          exists = true
          returnval = callback(null, key)
        }
      })

      if (exists) {
        return returnval
      }

      /* We've looped over the entire storage but not found our ID, let's add it */
      var newKey = KEY()
      storage.setItem(newKey, ID, function (err) {
        if (err) {
          returnval = callback(err)
        }
        returnval = callback(null, newKey)
      })

      return returnval
    }
  }
}

And my testfile:

var module = require('./index.js')('./tokenstorage', 100)

module.empty(function (err) {
  if (err) throw err
  setInterval(function () {
    module.getKey('myfirstID', function (err, KEY) {
      if (err) throw err
      console.log('KEY: ' + KEY)
    })
  }, 1000)
})

Issue with empty directory, node-persist crashing

I installed node-persist via npm and it continuously crashed on me (node.js 0.10 and 0.12 on Debian).

app.js- path.js:471
app.js-       throw new TypeError('Arguments to path.join must be strings');
app.js-             ^
app.js- TypeError: Arguments to path.join must be strings
app.js-     at Object.posix.join (path.js:471:13)
app.js-     at parseDirSync (/node/node_modules/node-persist/node-persist.js:681:49)
app.js-     at parseDataDirSync (/node/node_modules/node-persist/node-persist.js:617:12)
app.js-     at Object.exports.initSync (/node/node_modules/node-persist/node-persist.js:114:5)

I found a potential issue in line 678 in file node-persist/node-persist.js

        var arr = fs.readdirSync(dir);
        for (var i in arr) {
            var curr = arr[i];

If arr is empty, then "i" becomes "clean" (if you print it) and curr goes crazy and later path.join crashes.

I fixed it with:

        for (var i=0; i<arr.length; i++) {

export a storage factory

I know that this library is meant to simulate localStorage, but one feature that would make it more flexible is the ability to create many of them, (which can be initialized using different options).

Module throws errors

Two of your helper functions throw errors. This can cause apps that use your library to crash.

Since those are not API methods and are not documented as possibly throwing errors, then I'd recommend logging the error and moving on, rather than opening the possibility of crashing. :)

Attempting to throw non-existent error

I just ran into this error:

/home/sven/projects/pdfy/node_modules/node-persist/node-persist.js:538
            callback(err, result);
                     ^
  ReferenceError: err is not defined

  - node-persist.js:538 
    [pdfy]/[node-persist]/node-persist.js:538:22

The responsible bit of code is here - it doesn't appear to set err anywhere, thus it has nothing to throw.

Set TTL

Is there any way to set a time to live value?

Illegal operation on a directory error

What's this one about? I haven't specified a dir or anything - just using your sample code:

Error: EISDIR: illegal operation on a directory, open 'persist'
at Error (native)
at Object.fs.openSync (fs.js:549:18)
at Object.fs.writeFileSync (fs.js:1155:15)
at Object.LocalStorage.persistKeySync (/websites/demoit/node_modules/node-persist/local-storage.js:457:12)
at Object.LocalStorage.setItemSync (/websites/demoit/node_modules/node-persist/local-storage.js:275:14)
at Object.nodePersist.setItemSync (/websites/demoit/node_modules/node-persist/node-persist.js:99:29)
at Promise. (/websites/demoit/server.js:269:9)
at Promise. (/websites/demoit/node_modules/mongoose/node_modules/mpromise/lib/promise.js:177:8)
at emitOne (events.js:77:13)
at Promise.emit (events.js:169:7)
at Promise.emit (/websites/demoit/node_modules/mongoose/node_modules/mpromise/lib/promise.js:84:38)
at Promise.fulfill (/websites/demoit/node_modules/mongoose/node_modules/mpromise/lib/promise.js:97:20)
at Promise.resolve (/websites/demoit/node_modules/mongoose/lib/promise.js:114:23)
at Promise. (/websites/demoit/node_modules/mongoose/node_modules/mpromise/lib/promise.js:177:8)
at emitOne (events.js:77:13)
at Promise.emit (events.js:169:7)

this._expiredKeysInterval prevents node process from closing

If you call node-persist with the default options the handle for this._expiredKeysInterval prevents the node process from exiting normally.

You can call nodePersist.initSync({expiredInterval: false}) to avoid this but I don't think it's in the docs.

if continuous=false promise never resolves/rejects

storage.initSync({continuous: false});
storage.setItem('batman', "yo").then(
    function() {
        console.log("success");
        console.log(storage.getItem('batman'));
    },
    function() {
        console.log("error");
    }
);

setTimeout(function() {
    console.log("3 seconds later");
    storage.persist();  <-- suppose to resolve here.
}, 3000);

How to persists across commits (on Heroku)

Sorry for a newbie question: So I tried to use node-persist to store a really simple thing - pageviews because I do not want to setup a database for this. Then, I noticed that whenever I commit and push changes onto heroku. My page views get reset. After google around everyone is saying that Heroku couldn't store anything between commits and you have to use a db.... then, what's the point of node-persist? Is there an easy way to keep the data persisted across the commits without setting up db?

Deleting Storage Folder throws error on first start

Hello,

Not sure if this is a Problem of node-persist or Node but when I delete the Storage Folder a ENOENT error is thrown on first start.

Error: ENOENT: no such file or directory, open 'd:\path\to\project\projectname\server\storage\hash'

It fixes itself if you just restart the app but It cripples my deployment script. Is there a Cache File somewhere I need to delete?

Thanks for the Help

init() resolves before loading values

create a storage/persist/batman file and put a value in it. then.

storage.init({continuous: true})
    .then(function() {
        console.log("loaded");
        console.log(storage.getItem('batman'));
    });

Regex breaks on windows machine due to backslash.

Suggestion: String replace 53 with: btoaPathSepRegExp = new RegExp(path.sep.replace('','\'), 'g')

SyntaxError: Invalid regular expression: /\/: \ at end of pattern at new RegExp (native) at Object.<anonymous> (C:\Users\lukee_000\WebstormProjects\services\ShopifyService\node_modules\node-persist\src\local-storage.js:53:25) at Module._compile (module.js:413:34) at Object.Module._extensions..js (module.js:422:10) at Module.load (module.js:357:32) at Function.Module._load (module.js:314:12) at Module.require (module.js:367:17) at require (internal/module.js:16:19) at Object.<anonymous> (C:\Users\lukee_000\WebstormProjects\services\ShopifyService\node_modules\node-persist\src\node-persist.js:6:20) at Module._compile (module.js:413:34)

Question about child processes

This is my assumption, but I wanted to ask anyway: I'm assuming that when you run initSync() that you get a read from disk into memory. And then if you execute getItem, it's from memory, not disk?

In other words, if I have a forked child process and point it to the same directory with node-persist, and call setItem in process A, I probably cannot retrieve that with getItem in process B? Because the data is actually read from the in-process memory?

License & NPM

This is a great tool, would you mind adding a license? Most node stuff is MIT I believe.

Also great if you could publish to NPM.

[Question] nested values?

Hey, quick question..

Does your module allow for nested values?
For example I want to store something like this:

{
   key1: {
      child1: itsObject,
      child2: itsObject
      // Etc...
   },
   key2: {
      child1: itsObject,
      child2: itsObject
      // Etc...
   },
   // Etc....
}

Could I go

storage.setItem([key1][child1], itsObject);

Or would that return the following?

{
"[key1][child1]": itsObject
}

License Relese

Hi,

Do you plan to release it? MIT...
If yes when ?

Regards,
Shimi

async getItem promise

I would like to know why it's impossible to return a promise for getItem.

function getItemAsync(key) {
  return new Promise((resolve, reject) => {
    storage.getItem(key, (value, error) => {
      if (error) reject(error);
      else resolve(value);
    });
  });
}

Seems to do the job for me.

Special character in windows

Keys with special characters (\ / * ? " <> | ) fails to create the file entry in windows since its not allowed.

npm publish

Hey Simon, could you possibly do a npm publish on the repo to update the fixes to the breaking issues?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.