Question about `flockSync()`

flash-store

FlashStore is Key-Value persistent storage with easy to use ES6 Map-like API(both Async and Sync support), powered by LevelDB and TypeScript.

Requirements

Node.js v10 or above

Examples

Try the following command

npm install
npm run demo

The basic function as follows:

import { FlashStore } from 'flash-store'

const flashStore = new FlashStore('flashstore.workdir')

await flashStore.set(1, 'a')
console.log(`key: 1, value: ${await flashStore.get(1)}`)
// Output: 'a'

await flashStore.del(1)
console.log(`key: 1, value: ${await flashStore.get(1)}`)
// Output: undefined

Supported Backend

Backend	Flash Store	Install NPM Command
LevelDB	v1.0	`npm install flash-store@1`
SQLite	v0.20	`npm install [email protected]`
Medea	v0.18	~~npm install [email protected]~~ (deprecated)
SnapDB	v0.16	~~npm install [email protected]~~ (deprecated)
RocksDB	v0.14	`npm install [email protected]`
LevelDB	v0.12	`npm install [email protected]`

SnapDB & Medea were all deprecated because of lots of unknown bugs.

API Reference

`FlashStore`

FlashStore implements the Standard ES6 Map API with Async modification, powered by async-map-like

/**
 * ES6 Map API with Async
 */
export interface AsyncMap<K = any, V = any> {
  [Symbol.asyncIterator]() : AsyncIterableIterator<[K, V]>
  size                     : Promise<number>

  clear   ()                 : Promise<void>
  delete  (key: K)           : Promise<void>
  entries()                  : AsyncIterableIterator<[K, V]>
  get     (key: K)           : Promise<V | undefined>
  has     (key: K)           : Promise<boolean>
  keys    ()                 : AsyncIterableIterator<K>
  set     (key: K, value: V) : Promise<void>
  values  ()                 : AsyncIterableIterator<V>
}

class FlashStore<K, V> implements AsyncMap<K, V> {}

`FlashStoreSync`

FlashStoreSync implements the Standard ES6 Map API:

class FlashStoreSync<K, V> implements Map<K, V> {}

You get a sync API at the cost of all your data have to be kept in memory.
The data will be async writing back to disk for persistent storage in background.
The performance of FlashStoreSync can be expected high because it's all in memory.

Document

See auto generated docs

ECMAScript 6: Maps - 2ality – JavaScript and more

Known Issues

The gte and lte in options do not work property. (#4)

Version History

master v1.3 (Sep 12, 2021)

ES Modules support (#93)
Remove default workdir in FlashStore constructor.

v1.0 (Aug 16, 2021) LevelDB v7.0

LevelDB v7.0.0 support.

v0.20 Apr 2020 SQLite as Backend

We hardcoded the key type to be string only in this version.
We decide to try better-sqlite3 as it claim is very fast.
The other alternates (would love to try in the future if necessary):
1. TypeScript: sqlite
2. WebAssembly: sql.js

v0.18 Feb 2019 - Medea as Backend

DEPRECATED: Due to #79 #74 and also it is very unstable in production as my experiences. (e.g. memory leak & block event loop)

Switch from SnapDB to MedeaDown

Medea is a persistent key-value storage library that runs everywhere Node runs.

"It is a pure JS implementation of leveldown and it's almost as fast." — @Raynos link

"The LevelDOWN-compatible wrapper for Medea really opens up the possibility to reuse the modules that have already been created around the LevelUP ecosystem." — @kevinswiber link

Known Issues: FlashStore 会写满磁盘的问题 #155

async function compact (store: FlashStore): Promise<void> {
  await store.size
  const db = (store as any).levelDb.db.db.db
  await new Promise((resolve, reject) => {
    db.compact((err: any) => {
      if (err) {
        return reject(err)
      }
      resolve()
    })
  })
}

v0.16 May 2019 - SnapDB as Backend

Switch from RocksDB to SnapDB #45
#50 has been fixed. ~~WARN: Do not use this version because it has known issues~~

v0.14 May 2019 - RocksDB as Backend

Switch from LevelDB to RocksDB #34

v0.12 Jan 2019 - LevelDB as Backend

Use LevelDB as backend to skip the compiling when install.
Using leveldb official typings from @types/

v0.7 Aug 2018 - Nosql-LevelDB as Backend

Use nosql-leveldb as backend to prevent segfault.

v0.6 Jul 2018

Upgrade to TypeScript 3.0

v0.4 Jun 2018

1. Refactor API to implement ES6 `Map` interface

Update the API to ES6 Map-like, the difference is that FlashStore is all async.

2. Add `FlashStoreSync` as a in-memory Write-back Cache for Flash-Store

Add a new class FlashStoreSync which is a in-memory full loaded Write-back Cache for Flash-Store:

Writes directly to cache, lazy writes to store.
Reads from cache, never read-miss because cache have the full data of the store which will never expire.
API of FlashStoreSync is the same as the ES6 Map

v0.2 Sep 2017

Init version, API is LevelDB-like.

FAQ

Q: What's the difference between the `flash-store` and `memory-card`

Short answer:

flash-store is for save data to local filesystem.
memory-card is for save data to a distributed network storage, it can be serialized/deserialized safely with the minimum payload transfered by design.

Long answer:

flash-store and memory-card are all built by @huan, and they are all follow the ES6 Map API.

flash-store is using a no-sql local file database to maximum the performance, it can be used as a local database, or a local cache for whatever you want to cache from other API.

memory-card is using a local file to store data in JSON format by default, however, it supports more distributed methods and it can be serialized/deserialized safely with the minimum payload for transfer the data between networks (for example, it can serialize itself to only include the redis config and then deserialize it to restore a MemoryCard instance with the same data backend). Learn more from it's repository at here

Author

Huan LI (李卓桓) [email protected]

Copyright & License

Code released under the Apache-2.0 License
Docs released under Creative Commons

	public get size (): Promise<number> {
	log.verbose('FlashStore', 'size()')

	/* eslint no-async-promise-executor: 0 */
	// TODO: is there a better way to count all items from the db?
	return new Promise<number>(async (resolve, reject) => {
	try {
	let count = 0
	for await (const _ of this) {
	count++
	}

huan / flash-store Goto Github PK

flash-store's Introduction

flash-store's People

Contributors

Stargazers

Watchers

Forkers

flash-store's Issues

The devDependency rollup was updated from 1.9.2 to 1.9.3.

Bug Fixes

Pull Requests

See

The devDependency @types/node was updated from 12.12.1 to 12.12.2.

The dependency snap-db was updated from 1.1.5 to 1.1.6.

The devDependency @types/node was updated from 13.5.1 to 13.5.2.

Before(v0.2)

After(since v0.4)

See

The devDependency ts-node was updated from 8.4.0 to 8.4.1.

The devDependency @types/node was updated from 10.11.0 to 10.11.1.

The devDependency rollup was updated from 1.27.6 to 1.27.7.

Bug Fixes

Pull Requests

The devDependency tstest was updated from 0.4.5 to 0.4.9.

Recommend Projects

Recommend Topics

Recommend Org

The devDependency rollup was updated from `1.9.2` to `1.9.3`.

The devDependency @types/node was updated from `12.12.1` to `12.12.2`.

The dependency snap-db was updated from `1.1.5` to `1.1.6`.

The devDependency @types/node was updated from `13.5.1` to `13.5.2`.

The devDependency ts-node was updated from `8.4.0` to `8.4.1`.

The devDependency @types/node was updated from `10.11.0` to `10.11.1`.

The devDependency rollup was updated from `1.27.6` to `1.27.7`.

The devDependency tstest was updated from `0.4.5` to `0.4.9`.