freshy969/kv-storage

A proposal for an async key/value storage API for the web

KV Storage

This document is an explainer for a potential future web platform feature, "KV storage" (short for "key/value storage"). It's similar to localStorage in utility, but much more modern, and layered on top of IndexedDB.

This feature was formerly known as "async local storage", but in #38, folks pointed out that since we don't intend to access the localStorage database itself, that name was misleading.

This feature would be implemented as a built-in module. It would also use IndexedDB as its backing store; see more on that below.

A full specification is also available.

At TPAC 2018, the Web Platform Working Group and the IndexedDB specification editors agreed that after incubation in the WICG, KV storage should graduate to be part of the W3C IndexedDB specification.

Sample code

import { storage } from "std:kv-storage"; // specifier prefix not final

(async () => {
  await storage.set("mycat", "Tom");
  console.assert(await storage.get("mycat") === "Tom");

  for await (const [key, value] of storage.entries()) {
    console.log(key, value);
  }
  // Logs "mycat", "Tom"

  await storage.delete("mycat");
  console.assert(await storage.get("mycat") === undefined);
})();

Motivation

Local storage is a well-known and well-loved API. It only has one problem: it's synchronous. This leads to terrible performance and cross-window synchronization issues.

The alternative is IndexedDB. IndexedDB is, however, quite hard to use. It has no simple key/value layer, instead requiring understanding concepts like database upgrades and transactions. Its API is also quite dated; it does not use promises, but instead IDBRequest objects with their onsuccess and onerror methods.

In the face of this, a cottage industry of solutions for "async key/value storage" have sprung up to wrap IndexedDB. Perhaps the most well-known of these is localForage, which copies the localStorage API directly.

After many years of convergence in library-space on this sort of solution, it's time to bring a simple async key/value storage solution out of the cold and into the web platform.

API

Map-like key/value pair API

Note that keys and values would be allowed to be any structured-serializable type (see #2 for more discussion).

set(key, value)

Sets the value of the entry identified by key to value. Returns a promise that fulfills with undefined once this is complete.

Note: setting an entry to have the value undefined is equivalent to deleting it. See discussion in #3.

get(key)

Returns a promise for the value of the entry identified by key, or undefined if no value is present.

delete(key)

Removes the entry identified by key, if it exists. Once this completes, returns a promise for undefined.

Note: this is equivalent to set(key, undefined). See discussion in #3.

clear()

Clears all entries. Returns a promise for undefined.

keys()

Returns an async iterator for all the stored keys, sorted in the underlying IndexedDB key order.

values()

Returns an async iterator for all the stored values, sorted to correspond with keys().

entries()

Returns an async iterator of [key, value] pairs, sorted to correspond with keys().

new StorageArea(): separate storage areas

We additionally expose a StorageArea constructor, which allows you to create an "isolated" storage area that is less likely to collide than using the default one:

import { storage, StorageArea } from "std:kv-storage";

(async () => {
  await storage.set("mycat", "Tom");
  console.assert(await storage.get("mycat") === "Tom");

  const otherStorage = new StorageArea("unique string");
  console.assert(await otherStorage.get("mycat") === undefined);
  await otherStorage.set("mycat", "Jerry");
  console.assert(await otherStorage.get("mycat") === "Jerry");
})();

This sort of API has precedent in localForage, which is notable since localForage otherwise sticks rather strictly to the localStorage API surface.

The scope of the default storage area would be per-realm. (Or more precisely, per module map, since it would be created whenever you imported the module.)

backingStore: falling back to IndexedDB

One of the great things about layering KV storage on top of IndexedDB is that, if the developer's code grows beyond the capabilities of a simple key/value store, they can easily transition to the full power of IndexedDB (such as using transactions, indices, or cursors), while reusing their database.

To facilitate this, we include an API that allows you to get a { database, store, version } object identifying the IndexedDB database and store within that database where a given StorageArea's data is being stored:

import { storage } from "std:kv-storage";
import { open as idbOpen } from "https://www.npmjs.com/package/idb/pretend-this-was-a-native-JS-module";

(async () => {
  await storage.set("mycat", "Tom");
  await storage.set("mydog", "Joey");

  const { database, store, version } = storage.backingStore;
  const db = await idbOpen(database, version);
  const tx = db.transaction(store, "readwrite");
  tx.objectStore(store).add("mycat", "Jerry");
  tx.objectStore(store).add("mydog", "Kelby");
  await tx.complete;
  await db.close();
})();

Impact

• Developers would be steered away from the perils of localStorage, toward this modern, attractive alternative.
• Applications would no longer need to ship complex IndexedDB logic, or the 9.5 KB localForage library.

localForage is downloaded ~227K times per week via npm.

Another notable library in this vein is idb-keyval, which is downloaded ~119K times per week via npm.

For comparison, Angular is downloaded ~2.314 million times per week, React ~6.312 million times per week, and Vue ~995K times per week.

(All of the above statistics are as of 2019-03-14.)