Giter Site home page Giter Site logo

paulmillr / scure-base Goto Github PK

View Code? Open in Web Editor NEW
100.0 4.0 12.0 962 KB

Secure, audited & 0-deps implementation of bech32, base64, base32, base16 & base58

Home Page: https://paulmillr.com/noble/#scure

License: MIT License

JavaScript 68.00% TypeScript 26.85% Rust 5.15%
base58 base64 bech32 bech32m rfc4648 base16 base-x encoding decoding

scure-base's Introduction

scure-base

Audited & minimal implementation of bech32, base64, base58, base32 & base16.

  • ๐Ÿ”’ Audited by an independent security firm
  • ๐Ÿ”ป Tree-shakeable: unused code is excluded from your builds
  • ๐Ÿ“ฆ ESM and common.js
  • โœ๏ธ Written in functional style, easily composable
  • ๐Ÿ’ผ Matches specs

Check out Projects using scure-base.

This library belongs to scure

scure โ€” audited micro-libraries.

Usage

npm install @scure/base

We support all major platforms and runtimes. The library is hybrid ESM / Common.js package.

import { base16, base32, base64, base58 } from '@scure/base';
// Flavors
import {
  base58xmr,
  base58xrp,
  base32nopad,
  base32hex,
  base32hexnopad,
  base32crockford,
  base64nopad,
  base64url,
  base64urlnopad,
} from '@scure/base';

const data = Uint8Array.from([1, 2, 3]);
base64.decode(base64.encode(data));

// Convert utf8 string to Uint8Array
const data2 = new TextEncoder().encode('hello');
base58.encode(data2);

// Everything has the same API except for bech32 and base58check
base32.encode(data);
base16.encode(data);
base32hex.encode(data);

base58check is a special case: you need to pass sha256() function:

import { createBase58check } from '@scure/base';
createBase58check(sha256).encode(data);

Alternative API:

import { str, bytes } from '@scure/base';
const encoded = str('base64', data);
const data = bytes('base64', encoded);

Bech32, Bech32m and Bitcoin

We provide low-level bech32 operations. If you need high-level methods for BTC (addresses, and others), use scure-btc-signer instead.

Bitcoin addresses use both 5-bit words and bytes representations. They can't be parsed using bech32.decodeToBytes. Instead, do something this:

const decoded = bech32.decode(address);
// NOTE: words in bitcoin addresses contain version as first element,
// with actual witnes program words in rest
// BIP-141: The value of the first push is called the "version byte".
// The following byte vector pushed is called the "witness program".
const [version, ...dataW] = decoded.words;
const program = bech32.fromWords(dataW); // actual witness program

Same applies to Lightning Invoice Protocol BOLT-11. We have many tests in ./test/bip173.test.js that serve as minimal examples of Bitcoin address and Lightning Invoice Protocol parsers. Keep in mind that you'll need to verify the examples before using them in your code.

Design rationale

The code may feel unnecessarily complicated; but actually it's much easier to reason about. Any encoding library consists of two functions:

encode(A) -> B
decode(B) -> A
  where X = decode(encode(X))
  # encode(decode(X)) can be !== X!
  # because decoding can normalize input

e.g.
base58checksum = {
  encode(): {
    // checksum
    // radix conversion
    // alphabet
  },
  decode(): {
    // alphabet
    // radix conversion
    // checksum
  }
}

But instead of creating two big functions for each specific case, we create them from tiny composamble building blocks:

base58checksum = chain(checksum(), radix(), alphabet())

Which is the same as chain/pipe/sequence function in Functional Programming, but significantly more useful since it enforces same order of execution of encode/decode. Basically you only define encode (in declarative way) and get correct decode for free. So, instead of reasoning about two big functions you need only reason about primitives and encode chain. The design revealed obvious bug in older version of the lib, where xmr version of base58 had errors in decode's block processing.

Besides base-encodings, we can reuse the same approach with any encode/decode function (bytes2number, bytes2u32, etc). For example, you can easily encode entropy to mnemonic (BIP-39):

export function getCoder(wordlist: string[]) {
  if (!Array.isArray(wordlist) || wordlist.length !== 2 ** 11 || typeof wordlist[0] !== 'string') {
    throw new Error('Worlist: expected array of 2048 strings');
  }
  return mbc.chain(mbu.checksum(1, checksum), mbu.radix2(11, true), mbu.alphabet(wordlist));
}

base58 is O(n^2) and radixes

Uint8Array is represented as big-endian number:

[1, 2, 3, 4, 5] -> 1*(256**4) + 2*(256**3) 3*(256**2) + 4*(256**1) + 5*(256**0)
where 256 = 2**8 (8 bits per byte)

which is then converted to a number in another radix/base (16/32/58/64, etc).

However, generic conversion between bases has quadratic O(n^2) time complexity.

Which means base58 has quadratic time complexity too. Use base58 only when you have small constant sized input, because variable length sized input from user can cause DoS.

On the other hand, if both bases are power of same number (like 2**8 <-> 2**64), there is linear algorithm. For now we have implementation for power-of-two bases only (radix2).

Security

The library has been independently audited:

The library was initially developed for js-ethereum-cryptography. At commit ae00e6d7, it was extracted to a separate package called micro-base. After the audit we've decided to use @scure NPM namespace for security.

Resources

Projects using scure-base

License

MIT (c) Paul Miller (https://paulmillr.com), see LICENSE file.

scure-base's People

Contributors

alexgleason avatar benjreinhart avatar filosottile avatar imcotton avatar jacogr avatar mahnunchik avatar micahzoltu avatar paulmillr avatar sambacha avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

scure-base's Issues

Optional Padding

RFC4648 states:

In some circumstances, the use of padding ("=") in base-encoded data is not required or used.

An example of a specification that prohibits padding characters is RFC8555:

... the base64url alphabet and MUST NOT include base64 padding characters ("=").

Therefore, it would be nice if the encoding/decoding functions provided an option to omit padding characters.

feature request: add bech32 descriptor checksum

I don't want to interfere with the code base but it'd be useful to have this added

adapted from bitcoin core:

const INPUT_CHARSET =
    "0123456789()[],'/*abcdefgh@:$%{}" +
    "IJKLMNOPQRSTUVWXYZ&+-.;<=>?!^_|~" +
    'ijklmnopqrstuvwxyzABCDEFGH`#"\\ ',
  CHECKSUM_CHARSET = "qpzry9x8gf2tvdw0s3jn54khce6mua7l";

export const addChecksum = (str: string) => `${str}#${descChecksum(str)}`;

function polymod(c: bigint, val: number): bigint {
  let c0 = Number((c >> 35n) & 31n);
  c = ((c & 0x7ffffffffn) << 5n) ^ BigInt(val);
  if (c0 & 1) c ^= 0xf5dee51989n;
  if (c0 & 2) c ^= 0xa9fdca3312n;
  if (c0 & 4) c ^= 0x1bab10e32dn;
  if (c0 & 8) c ^= 0x3706b1677an;
  if (c0 & 16) c ^= 0x644d626ffdn;
  return c;
}

function descChecksum(span: string): string {
  let c = 1n;
  let cls = 0;
  let clscount = 0;
  for (let ch of span) {
    let pos = INPUT_CHARSET.indexOf(ch);
    if (pos === -1) return "";
    c = polymod(c, pos & 31);
    cls = cls * 3 + (pos >> 5);
    if (++clscount === 3) {
      c = polymod(c, cls);
      cls = 0;
      clscount = 0;
    }
  }
  if (clscount > 0) c = polymod(c, cls);
  for (let j = 0; j < 8; ++j) c = polymod(c, 0);
  c ^= 1n;

  let ret = new Array(8).fill(" ");
  for (let j = 0; j < 8; ++j)
    ret[j] = CHECKSUM_CHARSET[Number((c >> (5n * BigInt(7 - j))) & 31n)];
  return ret.join("");
}

Source map errors are emitted during debug

Received in v1.1.6

While debugging our project this package throws source map errors in dev tools:

Could not read source map for node_modules/%40scure/base/lib/index.js: ENOENT: no such file or directory, open /node_modules/@scure/base/lib/index.js.map'

TypeScript, by default, does not generate source maps. If you want TypeScript to produce source maps along with the compiled JavaScript files, you need to set the sourceMap option to true in your tsconfig.

Which functions should be faster?

If your app has is bottlenecked because of scure-base, write a comment here.

The current architecture is cool and readable, but not the fastest one. For example, hex.decode is slower than in noble-hashes / noble-curves. hex decoding is not used anywhere here, but it is used in scure-btc-signer. base64 is also a bit slow, but so far has not been a bottleneck.

Tests for `convertRadix` and `convertRadix2`

I've used convertRadix function on the cashaddr implementation #24 and it converts 5-bit number correctly according to the checksum tests.

But I've faced with that simple test gives incorrect result:

const data = [1];

console.log(convertRadix2(data, 8, 5, true));
// [ 0, 4 ] - correct
console.log(convertRadix(data, 2 ** 8, 2 ** 5));
// [ 1 ] - incorrect

I thought that I could understand the behavior from tests, but there are no tests for these methods.

Is this my mistake in use or is the method not implemented correctly?

Feature request: add option to make base32 padding optional

Some third-party and/or standard libraries support optional padding when encoding and decoding.

User facing ids that look like the below are an eyesore.

> base32.encode(crypto.getRandomValues(new Uint8Array(16)))
'7CI63KSR5ANU3BKSLYUFTGYQPM======'

I'm wondering if you would be open to adding a {padding: false} option or doing something like #4 for base32 as well?

Publish to deno.land/x

I would be great if this could be published to deno.land/x. For my specific usages of @scure/base in the libs I do publish, we currently point to esm.sh/@scure/[email protected] so it certainly would help my little corner of the world.

Normally I would make a PR to add mod.ts with a simple export * from './index.ts' however since there are webhook setup in the repo as well, it is best left as an issue since the single file PR doesn't add that much value to the overall process.

Wrong usage of `/* @__PURE__ */` annotation

Hi.

I am not sure if this a bug, but I still want to mention it in case it is.

Rollup throws this warning:
image

For this part of the code:

export const createBase58check = /* @__PURE__ */ (

It probably should be /* @__NO_SIDE_EFFECTS__ */ instead, right ?

Thank you.

@rollup/plugin-commonjs freaks out over code comments

Hello. This is not a critical bug, but I wanted to report it anyway since it started happening to me recently.

When trying to bundle my projects using rollup and @rollup/plugin-commonjs, I get the following warning:

[!] (plugin commonjs--resolver) Error: node_modules/@scure/base/lib/esm/index.js (343:27) A comment

"/* @__PURE__ */"

in "node_modules/@scure/base/lib/esm/index.js" contains an annotation that Rollup cannot interpret due to the position of the comment. The comment will be removed to avoid issues.
    at onwarn (file:///home/cscott/Projects/lib/crypto-tools/rollup.config-1697525011054.mjs:14:35)
    at /home/cscott/Projects/lib/crypto-tools/node_modules/rollup/dist/shared/rollup.js:649:13
    at /home/cscott/Projects/lib/crypto-tools/node_modules/rollup/dist/shared/rollup.js:635:32
    at Object.logger [as onLog] (/home/cscott/Projects/lib/crypto-tools/node_modules/rollup/dist/shared/rollup.js:1127:9)
    at Module.log (/home/cscott/Projects/lib/crypto-tools/node_modules/rollup/dist/shared/rollup.js:14343:22)
    at Program.parseNode (/home/cscott/Projects/lib/crypto-tools/node_modules/rollup/dist/shared/rollup.js:6141:48)
    at new NodeBase (/home/cscott/Projects/lib/crypto-tools/node_modules/rollup/dist/shared/rollup.js:6042:14)
    at new Program (/home/cscott/Projects/lib/crypto-tools/node_modules/rollup/dist/shared/rollup.js:12282:9)
    at Module.setSource (/home/cscott/Projects/lib/crypto-tools/node_modules/rollup/dist/shared/rollup.js:14422:20)
    at ModuleLoader.addModuleSource (/home/cscott/Projects/lib/crypto-tools/node_modules/rollup/dist/shared/rollup.js:18640:20)


error Command failed with exit code 1.

I can safely ignore the warning as it doesn't break the bundling process. But it is an issue so I thought I would bring it up.

Bech32 library fails my BIP 173 test vectors

Hello Paul, I hope all is well with you!

I am running your bech32 library through a list of test vectors that I have lifted from Bitcoin BIP 173.

It passes most tests, but fails on a number of edge cases (such as weird version numbers).

I had previously been using another bech32 implementation written by Pieter Wuille. That implementation passes the test vectors and had been previously working okay, but it is not designed to handle longer bech32 strings.

The bech32 format is a mess, and I'm not sure if these failing test vectors are that important, but I'd like to hear your feedback.

P.S I converted the test vectors to JSON for easier use.

Pass vectors:

[
  ["BC1QW508D6QEJXTDG4Y5R3ZARVARY0C5XW7KV8F3T4", "751e76e8199196d454941c45d1b3a323f1433bd6"],
  ["tb1qrp33g0q5c5txsp9arysrx4k6zdkfs4nce4xj0gdcccefvpysxf3q0sl5k7", "1863143c14c5166804bd19203356da136c985678cd4d27a1b8c6329604903262"],
  ["BC1SW50QA3JX3S", "751e"],
  ["bc1zw508d6qejxtdg4y5r3zarvaryvg6kdaj", "751e76e8199196d454941c45d1b3a323"],
  ["tb1qqqqqp399et2xygdj5xreqhjjvcmzhxw4aywxecjdzew6hylgvsesrxh6hy", "000000c4a5cad46221b2a187905e5266362b99d5e91c6ce24d165dab93e86433"]
]

Fail vectors:

[
  ["an84characterslonghumanreadablepartthatcontainsthenumber1andtheexcludedcharactersbio1569pvx", "overall max length exceeded"],
  ["pzry9x0s0muk", "No separator character"],
  ["1pzry9x0s0muk", "Empty HRP"],
  ["x1b4n0q5v", "Invalid data character"],
  ["li1dgmt3", "Too short checksum"],
  ["A1G7SGD8", "checksum calculated with uppercase form of HRP"],
  ["10a06t8", "empty HRP"],
  ["1qzzfhee", "empty HRP"],
  ["bc1qw508d6qejxtdg4y5r3zarvary0c5xw7kv8f3t5", "Invalid checksum"],
  ["BC13W508D6QEJXTDG4Y5R3ZARVARY0C5XW7KN40WF2", "Invalid witness version"],
  ["bc1rw5uspcuh", "Invalid program length"],
  ["bc10w508d6qejxtdg4y5r3zarvary0c5xw7kw508d6qejxtdg4y5r3zarvary0c5xw7kw5rljs90", "Invalid program length"],
  ["bc1zw508d6qejxtdg4y5r3zarvaryvqyzf3du", "zero padding of more than 4 bits"],
  ["tb1qrp33g0q5c5txsp9arysrx4k6zdkfs4nce4xj0gdcccefvpysxf3pjxtptv", "Non-zero padding in 8-to-5 conversion"],
  ["bc1gmk9yu", "Empty data section"]
]

Unable to decode a bech32 encoded lightning invoice.

I am running the following code in nodejs (version 19+):

import { bech32 } from '@scure/base'

const test_enc  = 'lnbc1u1pjvy84epp5zrapr3w7tqelvjzwm0rwsac2ga79m982uruducydr2u6zwlhpasqhp5fe47lwjexge0lff7ru2g6757g35qajscy39hsz4dvqe97gnt3d3scqzzsxqyz5vqsp5ptv9dz544r5pxd3gkulqelakrtmx4nf47xw4mmm8a0u8j2up7mqs9qyyssqumxjespzkuwzdppw3hzkawgdedjyu2e0wnsk3t3y8g7mkpz49nn9rlrzsj07tz3hjnld80j749069puz9uanhr55p9ngw46cy2w295qpktsz9y'

const { bytes } = bech32.decodeToBytes(test_enc)
const decoded   = new TextDecoder().decode(bytes)

console.log('decoded:', decoded)

I get the following error:

Error: Non-zero padding: 32
    at convertRadix2 (./node_modules/@scure/base/lib/esm/index.js:175:15)
    at decode (./node_modules/@scure/base/lib/esm/index.js:210:36)
    at Object.decodeToBytes (./node_modules/@scure/base/lib/esm/index.js:360:40)

The invoice decodes normally using other decoding tools.

I'm not sure if this is a bug, or that I am using the library incorrectly. The above code works for smaller encoded strings.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.