oklog/ulid

Universally Unique Lexicographically Sortable Identifier (ULID) in Go

Universally Unique Lexicographically Sortable Identifier

A Go port of ulid/javascript with binary format implemented.

Background

A GUID/UUID can be suboptimal for many use-cases because:

• It isn't the most character efficient way of encoding 128 bits
• UUID v1/v2 is impractical in many environments, as it requires access to a unique, stable MAC address
• UUID v3/v5 requires a unique seed and produces randomly distributed IDs, which can cause fragmentation in many data structures
• UUID v4 provides no other information than randomness which can cause fragmentation in many data structures

A ULID however:

• Is compatible with UUID/GUID's
• 1.21e+24 unique ULIDs per millisecond (1,208,925,819,614,629,174,706,176 to be exact)
• Lexicographically sortable
• Canonically encoded as a 26 character string, as opposed to the 36 character UUID
• Uses Crockford's base32 for better efficiency and readability (5 bits per character)
• Case insensitive
• No special characters (URL safe)
• Monotonic sort order (correctly detects and handles the same millisecond)

Install

This package requires Go modules.

go get github.com/oklog/ulid/v2

Usage

ULIDs are constructed from two things: a timestamp with millisecond precision, and some random data.

Timestamps are modeled as uint64 values representing a Unix time in milliseconds. They can be produced by passing a time.Time to ulid.Timestamp, or by calling time.Time.UnixMilli and converting the returned value to uint64.

Random data is taken from a provided io.Reader. This design allows for greater flexibility when choosing trade-offs, but can be a bit confusing to newcomers.

If you just want to generate a ULID and don't (yet) care about details like performance, cryptographic security, etc., use the ulid.Make helper function. This function calls time.Now to get a timestamp, and uses a source of entropy which is process-global, pseudo-random, and monotonic.

fmt.Println(ulid.Make())
// 01G65Z755AFWAKHE12NY0CQ9FH

More advanced use cases should utilize ulid.New.

entropy := rand.New(rand.NewSource(time.Now().UnixNano()))
ms := ulid.Timestamp(time.Now())
fmt.Println(ulid.New(ms, entropy))
// 01G65Z755AFWAKHE12NY0CQ9FH

Care should be taken when providing a source of entropy.

The above example utilizes math/rand.Rand, which is not safe for concurrent use by multiple goroutines. Consider alternatives such as x/exp/rand. Security-sensitive use cases should always use cryptographically secure entropy provided by crypto/rand.

Performance-sensitive use cases should avoid synchronization when generating IDs. One option is to use a unique source of entropy for each concurrent goroutine, which results in no lock contention, but cannot provide strong guarantees about the random data, and does not provide monotonicity within a given millisecond. One common performance optimization is to pool sources of entropy using a sync.Pool.

Monotonicity is a property that says each ULID is "bigger than" the previous one. ULIDs are automatically monotonic, but only to millisecond precision. ULIDs generated within the same millisecond are ordered by their random component, which means they are by default un-ordered. You can use ulid.MonotonicEntropy or ulid.LockedMonotonicEntropy to create ULIDs that are monotonic within a given millisecond, with caveats. See the documentation for details.

If you don't care about time-based ordering of generated IDs, then there's no reason to use ULIDs! There are many other kinds of IDs that are easier, faster, smaller, etc. Consider UUIDs.

Commandline tool

This repo also provides a tool to generate and parse ULIDs at the command line.

go install github.com/oklog/ulid/v2/cmd/ulid@latest

Usage:

Usage: ulid [-hlqz] [-f <format>] [parameters ...]
 -f, --format=<format>  when parsing, show times in this format: default, rfc3339, unix, ms
 -h, --help             print this help text
 -l, --local            when parsing, show local time instead of UTC
 -q, --quick            when generating, use non-crypto-grade entropy
 -z, --zero             when generating, fix entropy to all-zeroes

Examples:

$ ulid
01D78XYFJ1PRM1WPBCBT3VHMNV
$ ulid -z
01D78XZ44G0000000000000000
$ ulid 01D78XZ44G0000000000000000
Sun Mar 31 03:51:23.536 UTC 2019
$ ulid --format=rfc3339 --local 01D78XZ44G0000000000000000
2019-03-31T05:51:23.536+02:00

Specification

Below is the current specification of ULID as implemented in this repository.

Components

Timestamp

• 48 bits
• UNIX-time in milliseconds
• Won't run out of space till the year 10889 AD

Entropy

• 80 bits
• User defined entropy source.
• Monotonicity within the same millisecond with ulid.Monotonic

Encoding

Crockford's Base32 is used as shown. This alphabet excludes the letters I, L, O, and U to avoid confusion and abuse.

0123456789ABCDEFGHJKMNPQRSTVWXYZ

Binary Layout and Byte Order

The components are encoded as 16 octets. Each component is encoded with the Most Significant Byte first (network byte order).

0                   1                   2                   3
 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                      32_bit_uint_time_high                    |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|     16_bit_uint_time_low      |       16_bit_uint_random      |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                       32_bit_uint_random                      |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                       32_bit_uint_random                      |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

String Representation

 01AN4Z07BY      79KA1307SR9X4MV3
|----------|    |----------------|
 Timestamp           Entropy
  10 chars           16 chars
   48bits             80bits
   base32             base32

Test

go test ./...

Benchmarks

On a Intel Core i7 Ivy Bridge 2.7 GHz, MacOS 10.12.1 and Go 1.8.0beta1

BenchmarkNew/WithCryptoEntropy-8      2000000        771 ns/op      20.73 MB/s   16 B/op   1 allocs/op
BenchmarkNew/WithEntropy-8            20000000      65.8 ns/op     243.01 MB/s   16 B/op   1 allocs/op
BenchmarkNew/WithoutEntropy-8         50000000      30.0 ns/op     534.06 MB/s   16 B/op   1 allocs/op
BenchmarkMustNew/WithCryptoEntropy-8  2000000        781 ns/op      20.48 MB/s   16 B/op   1 allocs/op
BenchmarkMustNew/WithEntropy-8        20000000      70.0 ns/op     228.51 MB/s   16 B/op   1 allocs/op
BenchmarkMustNew/WithoutEntropy-8     50000000      34.6 ns/op     462.98 MB/s   16 B/op   1 allocs/op
BenchmarkParse-8                      50000000      30.0 ns/op     866.16 MB/s    0 B/op   0 allocs/op
BenchmarkMustParse-8                  50000000      35.2 ns/op     738.94 MB/s    0 B/op   0 allocs/op
BenchmarkString-8                     20000000      64.9 ns/op     246.40 MB/s   32 B/op   1 allocs/op
BenchmarkMarshal/Text-8               20000000      55.8 ns/op     286.84 MB/s   32 B/op   1 allocs/op
BenchmarkMarshal/TextTo-8             100000000     22.4 ns/op     714.91 MB/s    0 B/op   0 allocs/op
BenchmarkMarshal/Binary-8             300000000     4.02 ns/op    3981.77 MB/s    0 B/op   0 allocs/op
BenchmarkMarshal/BinaryTo-8           2000000000    1.18 ns/op   13551.75 MB/s    0 B/op   0 allocs/op
BenchmarkUnmarshal/Text-8             100000000     20.5 ns/op    1265.27 MB/s    0 B/op   0 allocs/op
BenchmarkUnmarshal/Binary-8           300000000     4.94 ns/op    3240.01 MB/s    0 B/op   0 allocs/op
BenchmarkNow-8                        100000000     15.1 ns/op     528.09 MB/s    0 B/op   0 allocs/op
BenchmarkTimestamp-8                  2000000000    0.29 ns/op   27271.59 MB/s    0 B/op   0 allocs/op
BenchmarkTime-8                       2000000000    0.58 ns/op   13717.80 MB/s    0 B/op   0 allocs/op
BenchmarkSetTime-8                    2000000000    0.89 ns/op    9023.95 MB/s    0 B/op   0 allocs/op
BenchmarkEntropy-8                    200000000     7.62 ns/op    1311.66 MB/s    0 B/op   0 allocs/op
BenchmarkSetEntropy-8                 2000000000    0.88 ns/op   11376.54 MB/s    0 B/op   0 allocs/op
BenchmarkCompare-8                    200000000     7.34 ns/op    4359.23 MB/s    0 B/op   0 allocs/op

Prior Art

• ulid/javascript
• RobThree/NUlid
• imdario/go-ulid

ulid's Issues

Generated ulids should be unambigious

The encoded form of the ulid carries 2bits more than the binary form. In order to have comparable string representations, only the following values can be used for the right most (least significant) symbol
0,4,8,C,G,M,R,W

This library generates other values. I would not consider it a bug, but a potential improvement.

Timestamp problem

I'm not sure if the output format: 0000XSNJG0MQJHBF4QX1EFD6Y3 is correct from the example. The leading zeros don't look right when compared with ulid's generated in other languages. So I made a slight change to the demo:

var entropy := rand.New(rand.NewSource(time.Now().UnixNano()))
func ExampleULID() {
var ts uint64 = uint64(time.Now().UnixNano() / 1000000)
value, _ := ulid.New(ts, entropy))
fmt.Println(value.String())
// Output: 01B3QSN2GDP1G124W1Q8D4PD1N
}

This seems to match encoded times from other implementations.

How to use ulid with ent

How to use ulid with ent as ID?

How to upgrade?

When I fetch the v2 version my code keeps pointing to 1.3.1 version but the latest version seems to be 2.1.0

ulid organization

Hey! I'm thinking of creating a ulid org, with the spec, and a couple reference implementations. That would mean breaking out the current repo into a spec repo, and a JS implementation repo. Would you be interested in merging with the org for the go implementation?

Add Parse function with byte parameter

Is it possible to add a ParseByte function.
func Parse(ulid []byte) (id ULID, err error) { return id, parse(ulid, false, &id) }

Because i get my data as a byte array and have to cast it to string and then this function recast is to a byte array again.

If you want this i can also create a PR with this added function

Parsing ignores invalid characters

When I was looking into #9, I noticed that the Parse() and UnmarshalText() methods do not check the input against the Base32 character set. This means that all sorts of invalid text ULIDs will parse into a seemingly valid ULID.

Is this expected? It breaks some assumptions I had made when parsing text-encoded ULIDs from an untrusted source.

The unmarshalling code is borrowed from NUlid, but it has an extra check that is lacking in this library: https://github.com/RobThree/NUlid/blob/89f5a9fc827d191ae5adafe42547575ed3a47723/NUlid/Ulid.cs#L250-L252

Example

package main

import (
	"log"
	"github.com/oklog/ulid"
)

func main() {
	input := " !\"#$%&'()*+_-/[]|~~\\|<>?\x00"
	id, err := ulid.Parse(input)
	if err != nil {
		log.Fatalf("could not parse ULID: %v", err)
	}
	log.Printf("Parse(%q).String() == %q", input, id.String())
}

$ go run parseinvalid.go
2018/01/30 13:39:11 Parse(" !\"#$%&'()*+_-/[]|~~\\|<>?\x00").String() == "7ZZZZZZZZZZZZZZZZZZZZZZZZZ"

Actual behavior

ulid.Parse() returns a nil error for values like " !\"#$%&'()*+_-/[]|~~\\|<>?\x00".

Expected behavior

ulid.Parse() returns a new ulid.ErrInvalidCharacters error for any text input outside of ulid.Encoding (case insensitive).

Is this a safe way to use ulid concurrently with other libraries too?

I just asked this question on SO. I'm posting this here too both for me and for for future "researchers".

I'm trying to use for the first time the ulid package.

In their README they say:

Please note that rand.Rand from the math package is not safe for concurrent use.
Instantiate one per long living go-routine or use a sync.Pool if you want to avoid the potential contention of a locked rand.Source as its been frequently observed in the package level functions.

Can you help me understand what does this mean and how to write SAFE code for concurrent use with libraries such ent or gqlgen?

Example: I'm using the below code in my app to generate new IDs (sometimes even many of them in the same millisecond which is fine for ulid).

import (
  "math/rand"
  "time"

  "github.com/oklog/ulid/v2"
)

var defaultEntropySource *ulid.MonotonicEntropy

func init() {
  defaultEntropySource = ulid.Monotonic(rand.New(rand.NewSource(time.Now().UnixNano())), 0)
}

func NewID() string {
  return ulid.MustNew(ulid.Timestamp(time.Now()), defaultEntropySource).String()
}

Is this a safe way to use the package?

Can you help me better understand?

Don't have MarshalText() return error

Right now, you can use String() to get the string representation of the ID. However, if you don't want the overhead of casting the byte slice into a string, you have to use MarshalText(), which returns an error, that – as far as I can see – will always be nil, since the buffer size cannot be changed by the user and will always be set correctly.

For a better user experience, I would propose either making this change:

- func (id ULID) MarshalText() ([]byte, error)
+ func (id ULID) MarshalText() []byte

or to add a new Bytes() method, equal to String().

How about a MakeFromTime(time.Time) convenience function?

In my code I sometimes need to create a ulid from a specific timestamp. To do this I do

ulid.MustNew(ulid.Timestamp(now), ulid.DefaultEntropy())

I wonder if other people do this and if it would be worth having a somewhat simpler

ulid.MakeFromTime(now)

If you think this is worth it, I am happy to create a PR.

Thank you!

big bug: Not Sortable

var count int64
func Ulid() string {
return ulid.MustNew(ulid.Timestamp(time.Now()), ulid.Monotonic(rand.New(rand.NewSource(atomic.AddInt64(&count, 1))), 0)).String()
}

for that,I write some testing code:

now := time.Now()
id := ulid.MustNew(ulid.Timestamp(now), ulid.Monotonic(rand.New(rand.NewSource(1)), 0))
id2 := ulid.MustNew(ulid.Timestamp(now), ulid.Monotonic(rand.New(rand.NewSource(2)), 0))
fmt.Println( (id2.String() > id.String()) == true) // false
if (id2.String() > id.String()) == false {
t.Error("big bug")
}
I want to get a true, but it return a false to me.

but the following code give me a true:
now := time.Now()
id := ulid.MustNew(ulid.Timestamp(now), ulid.Monotonic(rand.New(rand.NewSource(99)), 0))
id2 := ulid.MustNew(ulid.Timestamp(now), ulid.Monotonic(rand.New(rand.NewSource(100)), 0))
fmt.Println( (id2.String() > id.String()) == true) // true
if (id2.String() > id.String()) == false {
t.Error("big bug")
}

Collision probability

Can the readme document the collision probability? A link to another page which has this info is also fine.

Propose change/addition to README

The command
env GOPATH=$(pwd) GO111MODULE=on go get -v github.com/oklog/ulid/v2/cmd/ulid
returns the error
go: go.mod file not found in current directory or any parent directory. 'go get' is no longer supported outside a module. To build and install a command, use 'go install' with a version, like 'go install example.com/cmd@latest' For more information, see https://golang.org/doc/go-get-install-deprecation or run 'go help get' or 'go help install'.

With go version go1.18.4 linux/amd64 Ubuntu 20.04.4 LTS using the command go install -v github.com/oklog/ulid/v2/cmd/ulid@latest successfully installs ulid

How to generate lexicographically sorted ULIDs multiple times per ms.

How should I set up the entropy to generate lexicographically sorted ULIDs for the first 2 cases?

package main

import (
	"fmt"
	"math/rand"
	"sort"
	"time"

	"github.com/oklog/ulid"
)

func main() {
	run(time.Nanosecond)
	run(time.Microsecond)
	run(time.Millisecond)
}
func run(wait time.Duration) {
	fmt.Printf("Generate 1 ULID every: %s\n", wait.String())
	ids := []string{}
	for i := 0; i < 10; i++ {
		seed := time.Now().UnixNano()
		source := rand.NewSource(seed)
		entropy := rand.New(source)
		id := ulid.MustNew(ulid.Timestamp(time.Now()), entropy).String()
		ids = append(ids, id)
		fmt.Println(id)
		time.Sleep(wait)
	}
	fmt.Printf("Results are sorted: %t\n", sort.StringsAreSorted(ids))
}

outputs:
Generate 1 ULID every: 1ns
01ERZDG1G7NW4J1P7PNF21JXA1
01ERZDG1G89PRKEBPM05KVGH33
01ERZDG1G81H82CT5EGZ1WMRR2
01ERZDG1G8Y5KQVK7E5AW36SQB
01ERZDG1G85BAPHN0Y6Z2MKWCV
01ERZDG1G8X3C1YHKZJWPK4GV7
01ERZDG1G86B8TDA870HJ5SDQ4
01ERZDG1G8ZD2T95ZPZHWB50A7
01ERZDG1G8NGSF1KF6BV6CJ0V0
01ERZDG1G8GG7424V95JBCD9J8
Results are sorted: false

Generate 1 ULID every: 1µs
01ERZDG1G8ZVYN3PY11F1MNED2
01ERZDG1G87J51GMWXF56Q83CE
01ERZDG1G8HSFHR1MHJPV3AFND
01ERZDG1G8CNCBKG78SZVH4FKB
01ERZDG1G8NEA7YM61XHYZS4E0
01ERZDG1G8ZCZG33XNFSAAJK6Q
01ERZDG1G86RQ7YMX04AQC9SDX
01ERZDG1G81BPV6VSVXZMAJTQK
01ERZDG1G8C5JR0NX3CWMRWH51
01ERZDG1G83TDQKWZ04KT2R313
Results are sorted: false

Generate 1 ULID every: 1ms
01ERZDG1G8YA34MS33FW73KSRD
01ERZDG1G97THVX1Y1G9P8E3CE
01ERZDG1GAFMSDP7GYPMWTVK7Q
01ERZDG1GCGKFJ8Y5QQT90NX3Z
01ERZDG1GDR8MTDQNQHQVXYXKT
01ERZDG1GE1PZ8QQ7W1GZWVYXE
01ERZDG1GFNPA4X0GA5BAGNBQC
01ERZDG1GG2A6MCCFK5BBE74CT
01ERZDG1GH5Q1GMN1RTDZYWVDM
01ERZDG1GK1HXRCHQHFNE1YX4V
Results are sorted: true

Empty value availability

Is there an "empty" value that would pass the validation? Something like uuid v4 has with 00000000-0000-4000-0000-000000000000.

Add the Encoder and Decoder for gob encoding

It would be interesting to add the capacity to be encoded and decoded using the gob encoding package.

panic: runtime error: slice bounds out of range

I keep getting panics from the underlying bufio.Reader when generating a new ulid.
This happens quite frequently now in a personal project of mine (https://github.com/GeorgeMac/adagio/tree/gm/recovery).

I have reproduced this with both v1.3.1 and v2 versions.

Is this to do with a lack of entropy?

agent_two_1  | panic: runtime error: slice bounds out of range
agent_two_1  |
agent_two_1  | goroutine 42 [running]:
agent_two_1  | bufio.(*Reader).Read(0xc0000ac2a0, 0xc0000a8fb6, 0xa, 0xa, 0x0, 0x0, 0x100000000000000)
agent_two_1  | 	/usr/local/go/src/bufio/bufio.go:234 +0x3c7
agent_two_1  | io.ReadAtLeast(0xbda320, 0xc0000ac2a0, 0xc0000a8fb6, 0xa, 0xa, 0xa, 0x0, 0xc0001fd518, 0x40db18)
agent_two_1  | 	/usr/local/go/src/io/io.go:310 +0x88
agent_two_1  | io.ReadFull(...)
agent_two_1  | 	/usr/local/go/src/io/io.go:329
agent_two_1  | github.com/oklog/ulid.(*monotonic).MonotonicRead(0xc0000ae3c0, 0x16c8faa24f1, 0xc0000a8fb6, 0xa, 0xa, 0xc0001fd578, 0x4b1c66)
agent_two_1  | 	/workspace/vendor/github.com/oklog/ulid/ulid.go:518 +0x8b
agent_two_1  | github.com/oklog/ulid.New(0x16c8faa24f1, 0xbda7a0, 0xc0000ae3c0, 0xc0001fd5c8, 0xc0001fd5c8, 0x43d9ed, 0xc00018e178)
agent_two_1  | 	/workspace/vendor/github.com/oklog/ulid/ulid.go:94 +0x125
agent_two_1  | github.com/oklog/ulid.MustNew(...)
agent_two_1  | 	/workspace/vendor/github.com/oklog/ulid/ulid.go:105
agent_two_1  | github.com/georgemac/adagio/pkg/worker.NewPool.func1(0xa351a0)
agent_two_1  | 	/workspace/pkg/worker/worker.go:62 +0x138
agent_two_1  | github.com/georgemac/adagio/pkg/worker.(*Pool).handleEvent(0xc0001e02a0, 0xc0000aff80, 0x2, 0x0)
agent_two_1  | 	/workspace/pkg/worker/worker.go:108 +0xa0
agent_two_1  | github.com/georgemac/adagio/pkg/worker.(*Pool).Run.func1(0xc0001de050, 0xc0001e02a0, 0xbea8c0, 0xc0000ae7c0)
agent_two_1  | 	/workspace/pkg/worker/worker.go:88 +0x19b
agent_two_1  | created by github.com/georgemac/adagio/pkg/worker.(*Pool).Run
agent_two_1  | 	/workspace/pkg/worker/worker.go:80 +0x96

UPDATE:

Expected Behaviour

Generation of a ulid to not lead to a bufio.ReadAll operation to panic.

Actual Behaviour

Concurrent use of ulid.MustNew(...) leads to the panic shown above.

Steps to Reproduce

package main

import (
	"fmt"
	"math/rand"
	"sync"
	"time"

	"github.com/oklog/ulid"
)

var entropy = ulid.Monotonic(rand.New(rand.NewSource(time.Now().UnixNano())), 0)

func main() {
	var wg sync.WaitGroup
	for i := 0; i < 1000; i++ {
		wg.Add(1)
		go func() {
			defer wg.Done()
			fmt.Println(ulid.MustNew(ulid.Timestamp(time.Now().UTC()), entropy))
		}()
	}
	wg.Wait()
}

Run the above (I realize now that it is actually when it is called concurrently)

See panic:

01DJ7VTPXP103NN85XXWR023FM
01DJ7VTPXP103NN85XZ9TC6G3S
panic: runtime error: slice bounds out of range

goroutine 37 [running]:
bufio.(*Reader).Read(0xc0000a40c0, 0xc00001c0e6, 0xa, 0xa, 0x0, 0x0, 0x100000000000000)
	/usr/local/Cellar/go/1.12.6/libexec/src/bufio/bufio.go:234 +0x3c7
io.ReadAtLeast(0x10e6860, 0xc0000a40c0, 0xc00001c0e6, 0xa, 0xa, 0xa, 0xc000054400, 0xc0000bc6d0, 0x100b0a8)
	/usr/local/Cellar/go/1.12.6/libexec/src/io/io.go:310 +0x88
io.ReadFull(...)
	/usr/local/Cellar/go/1.12.6/libexec/src/io/io.go:329
github.com/oklog/ulid.(*monotonic).MonotonicRead(0xc0000b4000, 0x16c8fbd5bb6, 0xc00001c0e6, 0xa, 0xa, 0xc0000bc730, 0x1081946)
	/Users/georgemac/go/pkg/mod/github.com/oklog/[email protected]/ulid.go:518 +0x8b
github.com/oklog/ulid.New(0x16c8fbd5bb6, 0x10e68a0, 0xc0000b4000, 0x0, 0x0, 0x0, 0x0)
	/Users/georgemac/go/pkg/mod/github.com/oklog/[email protected]/ulid.go:94 +0x125
github.com/oklog/ulid.MustNew(...)
	/Users/georgemac/go/pkg/mod/github.com/oklog/[email protected]/ulid.go:105
main.main.func1(0xc0000b0010)
	/Users/georgemac/github/georgemac/ulidpanic/main.go:20 +0x153
created by main.main
	/Users/georgemac/github/georgemac/ulidpanic/main.go:18 +0x78
exit status 2

Additional Context

> uname -a                                                                      acceptance:(twodotoh)
Darwin Georges-MacBook-Pro.local 18.6.0 Darwin Kernel Version 18.6.0: Thu Apr 25 23:16:27 PDT 2019; root:xnu-4903.261.4~2/RELEASE_X86_64 x86_64
> go version                                                                    acceptance:(twodotoh)
go version go1.12.6 darwin/amd64
> cat go.sum                                                                                                                                                                             
github.com/oklog/ulid v1.3.1 h1:EGfNDEx6MqHz8B3uNV6QAib1UR2Lm97sHi3ocA6ESJ4=
github.com/oklog/ulid v1.3.1/go.mod h1:CirwcVhetQ6Lv90oh/F+FBtV6XMibvdAFo93nm5qn4U=
github.com/oklog/ulid/v2 v2.0.2 h1:r4fFzBm+bv0wNKNh5eXTwU7i85y5x+uwkxCUTNVQqLc=
github.com/oklog/ulid/v2 v2.0.2/go.mod h1:mtBL0Qe/0HAx6/a4Z30qxVIAL1eQDweXq5lxOEiwQ68=
github.com/pborman/getopt v0.0.0-20170112200414-7148bc3a4c30/go.mod h1:85jBQOZwpVEaDAr341tbn15RS4fCAsIst0qp7i8ex1o=

A code for SAFE use across concurrent goroutines?

I'm trying to use for the first time the ulid package, but using the below code I'm getting panics everywhere.

Can you help me understand why and how to write code SAFE for concurrent use across goroutines?

REPL: https://go.dev/play/p/Ysr8cCgF44n

package main

import (
    // "math/rand"
    "crypto/rand"
    "fmt"
    "log"
    "sync"
    "time"

    "github.com/oklog/ulid/v2"
)

func main() {
    var wg sync.WaitGroup

    // entropy := ulid.Monotonic(rand.New(rand.NewSource(time.Now().UnixNano())), 0)
    entropy := ulid.Monotonic(rand.Reader, 0)

    workers := 5

    ids := make([]ulid.ULID, 0, 10)

    for w := 0; w < workers; w++ {
        wg.Add(1)

        go func(w int) {
            defer wg.Done()

            for i := 0; i < cap(ids)/workers; i++ {
                ids = append(ids, ulid.MustNew(ulid.Timestamp(time.Now()), entropy))
            }
        }(w)
    }

    wg.Wait()

    seen := make(map[ulid.ULID]bool)

    for _, id := range ids {
        if seen[id] {
            fmt.Println(id)
            log.Fatal("duplicate")
        }
        seen[id] = true
    }
}

Maybe a bad example in the README?

I'm not a big fan of the example in the README. If the same code block is execute 2x, the ID generated is the same. Perhaps an example that generates something more unique would be better?

big bug: ulid is alway not sortable

var list = make([]string, 0)
var monotonic = ulid.Monotonic(rand.New(rand.NewSource(time.Now().UnixNano())), 1)
var busy = make(chan bool, 1)

func AddUlid2List(index string) {
busy <- true
id, _ := ulid.New(ulid.Timestamp(time.Now()), monotonic)
list = append(list, id.String()+" "+index)
<-busy
}

func Test_test6(t *testing.T) {
wg := sync.WaitGroup{}
wg.Add(2)
go func() {
for i := 0; i < 1000000; i++ {
AddUlid2List("1")
}
wg.Done()
}()
go func() {
for i := 0; i < 1000000; i++ {
AddUlid2List("2")
}
wg.Done()
}()
wg.Wait()
for i := 0; i < len(list); i++ {
if i+1 < len(list) && list[i] > list[i+1] {
//63718 01GH13A2V6MBDBNBG8TSH0B0ZD 2 01GH13A2V5W3TXG0EX5GF9Q4MP 1
fmt.Println(i, list[i], list[i+1])
t.Error("big bug")
}
}
if list[len(list)-2] > list[len(list)-1] {
t.Error("big bug2")
fmt.Println(list[len(list)-2], list[len(list)-1])
}
fmt.Println("END")
}

func Test_ulid(t *testing.T) {
i := 0
for {
i++
Test_test6(t)
if i == 10 {
break
}
}
}

run the method Test_ulid, it print:
63718 01GH13A2V6MBDBNBG8TSH0B0ZD 2 01GH13A2V5W3TXG0EX5GF9Q4MP 1
big bug

New release with go.modules?

Is it possible to make a new release with go.modules support?

cc: @tsenart / @peterbourgon

Text ULID parsing squashes high bits

This issue was discovered by @larsyencken in valohai/ulid2#4; ulid2 uses the same decoding code as this library. Quoting that issue:

We discovered this problem by accident, when we realised that some (very far future ULIDs) the ULIDs appear to be not time-ordering.
For example, ULIDs that start with 0, 8, G or R are mapped to the same place.
It looks like the parsing of the first character has a cycle in it, instead of generating a new sequence of binary ULIDs.

I'm wondering whether this is a bug or a limitation of the encoding.

The same issue is reproducible using this library, too:

package main

import "fmt"
import "github.com/oklog/ulid"

func main() {
	fmt.Println(ulid.MustParse("00000000000000000000000000"))
	fmt.Println(ulid.MustParse("80000000000000000000000000"))
	fmt.Println(ulid.MustParse("G0000000000000000000000000"))
	fmt.Println(ulid.MustParse("R0000000000000000000000000"))
}

outputs

00000000000000000000000000
00000000000000000000000000
00000000000000000000000000
00000000000000000000000000

go get github.com/oklog/ulid/v2

getting this error: how to resolve when i did
go get github.com/oklog/ulid/v2

package github.com/oklog/ulid/v2: cannot find package "github.com/oklog/ulid/v2" in any of:
/usr/lib/go-1.10/src/github.com/oklog/ulid/v2 (from $GOROOT)
/root/go/src/github.com/oklog/ulid/v2 (from $GOPATH)

can you post some example on usage?

can you post some example on usage?

i'm interested in the return value of string etc

bench testing panic

environment

OS: Windows 10
go version: go1.19.5 windows/amd64

code detail

my code(simple_id.go):

package simple_id

import (
	"github.com/oklog/ulid/v2"
	"math/rand"
	"sync"
	"time"
)

func SomeIds(count int) (ids []string) {
	ids = make([]string, count, count)
	for i := 0; i < count; i++ {
		ids[i] = ulid.Make().String()
	}
	return ids
}

func SomeIdsParallel(count int) (ids []string) {
	ids = make([]string, count, count)
	var wg sync.WaitGroup
	wg.Add(count)
	entropy := rand.New(rand.NewSource(time.Now().UnixNano()))
	ms := ulid.Timestamp(time.Now())
	for i := 0; i < count; i++ {
		go func(idx int) {
			defer wg.Done()
			if id, err := ulid.New(ms, entropy); err != nil {
				// log.Printf("count %v, idx %v, error %+v", count, idx, err)
				return
			} else {
				ids[idx] = id.String()
			}
		}(i)
	}
	wg.Wait()
	return ids
}

my testing code(simple_id_test.go):

package simple_id

import (
	"testing"
)

func TestSomeIds(t *testing.T) {
	type args struct {
		count int
	}
	tests := []struct {
		name    string
		args    args
		wantIds []string
	}{
		{"3 ids", args{3}, []string{}},
		{"10 ids", args{10}, []string{}},
	}
	for _, tt := range tests {
		t.Run(tt.name, func(t *testing.T) {
			gotIds := SomeIds(tt.args.count)
			for i, id := range gotIds {
				t.Logf("tests = %v, SomeIds() index = %v, id = %v", tt.name, i, id)
			}
		})
	}
}

func BenchmarkSomeIdsParallel(b *testing.B) {
	type args struct {
		count int
	}
	tests := []struct {
		name    string
		args    args
		wantIds []string
	}{
		{"3 ids", args{3}, []string{}},
		{"10 ids", args{10}, []string{}},
		{"100 ids", args{100}, []string{}},
	}
	b.ResetTimer()
	for i := 0; i < b.N; i++ {
		for _, tt := range tests {
			b.RunParallel(func(pb *testing.PB) {
				for pb.Next() {
					SomeIdsParallel(tt.args.count)
				}
			})
		}
	}
}

run command an powershell

go test -bench="BenchmarkSomeIdsParallel" -benchmem -count=4

The test may succeed or fail

success:

PS E:\code\GoglandProjects\test-any\simple_id> go test -bench="BenchmarkSomeIdsParallel" -benchmem -count=4
goos: windows
goarch: amd64
pkg: test-any/simple_id
cpu: Intel(R) Core(TM) i7-6700HQ CPU @ 2.60GHz
BenchmarkSomeIdsParallel-8           338          10665132 ns/op        10802996 B/op     157918 allocs/op
BenchmarkSomeIdsParallel-8           360          11162406 ns/op        11505852 B/op     168191 allocs/op
BenchmarkSomeIdsParallel-8           349          10791613 ns/op        11154578 B/op     163056 allocs/op
BenchmarkSomeIdsParallel-8           357          11379811 ns/op        11410379 B/op     166794 allocs/op
PASS
ok      test-any/simple_id      16.881s

fail:

PS E:\code\GoglandProjects\test-any\simple_id> go test -bench="BenchmarkSomeIdsParallel" -benchmem -count=4
goos: windows
goarch: amd64
pkg: test-any/simple_id
cpu: Intel(R) Core(TM) i7-6700HQ CPU @ 2.60GHz
BenchmarkSomeIdsParallel-8           337          11598445 ns/op        10771763 B/op     157458 allocs/op
BenchmarkSomeIdsParallel-8      panic: runtime error: index out of range [-1]                         
goroutine 23572324 [running]:
math/rand.(*rngSource).Uint64(...)
        C:/Program Files/Go/src/math/rand/rng.go:249
math/rand.(*rngSource).Int63(...)
        C:/Program Files/Go/src/math/rand/rng.go:234
math/rand.read({0xc0006ad386, 0xa, 0x78a5cd?}, {0x8eea78?, 0xc00091e000?}, 0xc0000b7e50, 0xc0000b7e58)
        C:/Program Files/Go/src/math/rand/rand.go:274 +0x176
math/rand.(*Rand).Read(0xc0001e04e0?, {0xc0006ad386?, 0x78b4dd?, 0x89f940?})
        C:/Program Files/Go/src/math/rand/rand.go:264 +0x65
io.ReadAtLeast({0x8ee680, 0xc0000b7e30}, {0xc0006ad386, 0xa, 0xa}, 0xa)
        C:/Program Files/Go/src/io/io.go:332 +0x9a
io.ReadFull(...)
        C:/Program Files/Go/src/io/io.go:351
github.com/oklog/ulid/v2.New(0x1862f0ab7fa, {0x8ee680?, 0xc0000b7e30})
        C:/Users/lidawei/go/pkg/mod/github.com/oklog/ulid/[email protected]/ulid.go:109 +0x11c
test-any/simple_id.SomeIdsParallel.func1(0x2)
        E:/code/GoglandProjects/test-any/simple_id/simple_id.go:27 +0x9b
created by test-any/simple_id.SomeIdsParallel
        E:/code/GoglandProjects/test-any/simple_id/simple_id.go:25 +0x270
exit status 2
FAIL    test-any/simple_id      7.049s

Data race on MonotonicRead

I got some data race warnings when running go test -race on some code using this library.

Should the documentation be improved and do I need to use a mutex?

WARNING: DATA RACE
Read at 0x00c4200aa8a0 by goroutine 8:
.../github.com/oklog/ulid.(*monotonic).MonotonicRead()

Previous write at 0x00c4200aa8a0 by goroutine 10:
.../github.com/oklog/ulid.(*uint80).SetBytes()

Getting duplicate ID's

I'm using this library to generate ULID's for a primary key in a database and when I run my benchmarks I'm finding a lot of duplicate primary key errors from my database. I'm assuming I'm using this library wrong because. What is the guidance on how to use this to avoid generating duplicate keys?

I'm doing the following.

now := time.Now()
entropy := ulid.Monotonic(rand.New(rand.NewSource(now.UnixNano())), 0)

transaction, err := pool.Begin()

for i := 0; i < b.N; i++ {
    id := ulid.MustNew(ulid.Timestamp(now), entropy)
    
    // INSERT SQL
    tx.Exec(INSERT_SQL_USING_id)
}

tx.Commit()

What have I done wrong that generates duplicate ULID's?

`go test -bench=.' is failing

env:

GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH=???
GORACE=""
GOROOT="/usr/local/Cellar/go/1.7.4/libexec"
GOTOOLDIR="/usr/local/Cellar/go/1.7.4/libexec/pkg/tool/darwin_amd64"
CC="clang"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/xd/pgmdc_gx2gd0m459yxc5gdr00000gn/T/go-build114248060=/tmp/go-build -gno-record-gcc-switches -fno-common"
CXX="clang++"
CGO_ENABLED="1"

🚀: (master)>go test -bench=.
--- FAIL: TestTimestamp (0.00s)
ulid_test.go:287: got timestamp 4773815605011, want 281474976710655
--- FAIL: TestParseRobustness (0.00s)
ulid_test.go:245: runtime error: index out of range
ulid_test.go:260: #1: failed on input [26]uint8{0x1, 0xc0, 0x73, 0x62, 0x4a, 0xaf, 0x39, 0x78, 0x51, 0x4e, 0xf8, 0x44, 0x3b, 0xb2, 0xa8, 0x59, 0xc7, 0x5f, 0xc3, 0xcc, 0x6a, 0xf2, 0x6d, 0x5a, 0xaa, 0x20}
FAIL
exit status 1
FAIL _/Users/hsawhney/code/ulid 4.985s
🚀: (master)>

Proposal: Convenience Comparison Methods to ULID type

Hello everyone,

I've noticed that while we have a Compare method for comparing two ULID instances, it might be beneficial to have additional convenience methods for specific comparisons. Specifically, I propose implementing methods like Equal, Less, LessEq, Greater, and GreaterEq.

These methods would essentially be wrappers around the existing Compare method, providing a more intuitive and readable interface for common comparison operations. Here's a rough idea of what these methods could look like:

func (u ULID) Equal(other ULID) bool {
  return u.Compare(other) == 0
}

func (u ULID) Less(other ULID) bool {
  return u.Compare(other) < 0
}

func (u ULID) LessEq(other ULID) bool {
  return u.Compare(other) <= 0
}

func (u ULID) Greater(other ULID) bool {
  return u.Compare(other) > 0
}

func (u ULID) GreaterEq(other ULID) bool {
  return u.Compare(other) >= 0
}

I believe these convenience methods would make the library more user-friendly by making the code more readable and reducing the cognitive load on developers.

x := ulid.Make()
y := ulid.Make()

if x.Compare(y) <= 0 {
    // do something
}

if x.LessEq(y) {
    // do something
}

I would love to hear your thoughts on this proposal. If there's an agreement, I'm more than willing to work on the implementation and submit a PR.

Best regards,
Patrick

ULIDs contains `ILOU` will be parsed as weird timestamps

Hi! I'm writing a new Ruby library for handling ULID in these days.
Now I’m testing other implementations examples in kachick/ruby-ulid#53.

And I have found weird examples in original repository as ulid/javascript#85.

And then checked the parser of this library, because I'm using this in a Go project, it is so useful! 😄

Using this command line tool as below, the version is https://github.com/oklog/ulid/tree/e7ac4de44d238ff4707cc84b9c98ae471f31e2d1

$ ulid -h
Usage: ulid [-hlqz] [-f <format>] [parameters ...]
 -f, --format=<format>  when parsing, show times in this format: default, rfc3339, unix, ms
 -h, --help             print this help text
 -l, --local            when parsing, show local time instead of UTC
 -q, --quick            when generating, use non-crypto-grade entropy
 -z, --zero             when generating, fix entropy to all-zeroes

$ ulid 01111111111111111111111111
Mon Dec 19 08:09:04.801 UTC 2005

$ ulid 0LLLLLLLLLLLLLLLLLLLLLLLLL # `L` is same as `1` in https://www.crockford.com/base32.html, but returned different value
Tue Aug 02 05:31:50.655 UTC 10889

$ ulid 0UUUUUUUUUUUUUUUUUUUUUUUUU # `U` is invalid in https://www.crockford.com/base32.html, but does not raise error
Tue Aug 02 05:31:50.655 UTC 10889

$ ulid 00000000000000000000000000
Thu Jan 01 00:00:00 UTC 1970

$ ulid 0OOOOOOOOOOOOOOOOOOOOOOOOO # `O` is same as `0` in https://www.crockford.com/base32.html, but returned different value
Tue Aug 02 05:31:50.655 UTC 10889

In my understanding, Crockford's base32 does not contain L I O for the encoded product. So I think ULID can handle them as invalid values 🤔 ref: ulid/spec#38, kachick/ruby-ulid#57

UUID compatibility

Does it possible to add function that returns uuid compat 36 character string?
I have some software that needs 36 character string like uuid ([]byte, and string representation with -)

No need to set timezone

I think unix timestamp does not depend on the timezone so there is no need to set it here: https://github.com/oklog/ulid/blob/master/ulid.go#L397

https://play.golang.org/p/AbtZT5gO7Vt:

package main

import (
	"fmt"
	"time"
)

func main() {
	x1 := time.Now().UTC()
	location, err := time.LoadLocation("America/New_York")
	if err != nil {
		panic(err)
	}
	x2 := x1.In(location)

	fmt.Printf("%s != %s\n", x1, x2)
	fmt.Printf("%d == %d\n", x1.Unix(), x2.Unix())
	fmt.Printf("%d == %d\n", x1.Nanosecond(), x2.Nanosecond())

}

prints:

2009-11-10 23:00:00 +0000 UTC != 2009-11-10 18:00:00 -0500 EST
1257894000 == 1257894000
0 == 0

Locally I get (because play.golang.org has some fancy clock):

2020-11-04 05:49:17.319471 +0000 UTC == 2020-11-04 00:49:17.319471 -0500 EST
1604468957 == 1604468957
319471000 == 319471000

Why not make MonotonicRead thread safe?

It's easy to wrap it into a lock, isn't it? Or at least add such an option if speed is an issue. I could make a PR if it's ok with you

Error Unmarshaling from sqlx queries

type Team struct {
	ID         ulid.ULID `db:"id"`
	Name       string    `db:"name"`
	Status     string    `db:"status"`
	CreatedAt  time.Time `db:"created_at"`
	UpdatedAt  time.Time `db:"updated_at"`
	Archivedat time.Time `db:"archived_at"`
}

Usage:

	query := "SELECT * FROM teams WHERE status = 'active'"
	teams := []Team{}
	if err := r.DbConn.SelectContext(ctx, &teams, query); err != nil {
		return nil, errors.Wrap(err, "selecting teams")
	}

Result:
error retrieving data selecting teams: sql: Scan error on column index 0, name "id": ulid: bad data size when unmarshaling

Table Structure:

create table teams (
        id varchar(26) not null,
        name varchar(255) not null,
        status varchar(26) not null,
        created_at datetime not null default now(),
        updated_at datetime not null default now(),
        archived_at datetime null
      ) default character set utf8mb4 engine = InnoDB;

Generation of new ULID is not thread safe

In order to reproduce, write tests that execute in parallel (or spawn multiple routines) that attempt to create a new ULID (MustNew / New) at the same time.
Eventually it will fail with DATA RACE error.

Read at 0x00c0000d9500 by goroutine 9:
math/rand.(*rngSource).Int63()
/usr/local/go/src/math/rand/rng.go:239 +0x3e
math/rand.(*Rand).Int63()
/usr/local/go/src/math/rand/rand.go:85 +0x50
math/rand.(*Rand).Int63-fm()
/usr/local/go/src/math/rand/rand.go:264 +0x41
math/rand.read()
/usr/local/go/src/math/rand/rand.go:272 +0xcb
math/rand.(*Rand).Read()
/usr/local/go/src/math/rand/rand.go:264 +0x159
/vendor/github.com/oklog/ulid.New()
/usr/go-workspace/src//vendor/github.com/oklog/ulid/ulid.go:82 +0x12a
/vendor/github.com/oklog/ulid.MustNew()
/usr/go-workspace/src//vendor/github.com/oklog/ulid/ulid.go:91 +0x61
/infrastructure/id.(*ulidHelper).GenerateID()
/usr/go-workspace/src//infrastructure/id/ulid_helper.go:36 +0x130
/infrastructure/id.TestIsValidId_WhenIdIsGenerated_ReturnsTrue()
/usr/go-workspace/src//infrastructure/id/ulid_helper_test.go:47 +0x4a
testing.tRunner()
/usr/local/go/src/testing/testing.go:827 +0x162

Duplicate ID's

I'm adding to the DB in very quick succession around 1000 rows.

I decided to output the ULIDs so I could catch if there were dupes:

Here are the relevant parts:

01F9VG0NT0ACNSHJM2J4G6WH3E
01F9VG0NT0ACNSHJM2J8WRTTJE
01F9VG0NT0ACNSHJM2JAC9W6YT
01F9VG0NT0ACNSHJM2JDWKQS65
01F9VG0NT0ACNSHJM2J8KV2SB2
01F9VG0NT0ACNSHJM2J4G6WH3E
Error 1062: Duplicate entry '01F9VG0NT0ACNSHJM2J4G6WH3E' for key 'PRIMARY'

01F9VG0NT0ACNSHJM2N53XTXJH
01F9VG0NT0ACNSHJM2N8BWJXPF
01F9VG0NT0ACNSHJM2N8QXRBDM
01F9VG0NT0ACNSHJM2N53XTXJH
Error 1062: Duplicate entry '01F9VG0NT0ACNSHJM2N53XTXJH' for key 'PRIMARY'

01F9VG0NT0ACNSHJM2SMR13HZ2
01F9VG0NT0ACNSHJM2SQVZW987
01F9VG0NT0ACNSHJM2SVQJ87QQ
01F9VG0NT0ACNSHJM2SMR13HZ2
Error 1062: Duplicate entry '01F9VG0NT0ACNSHJM2SMR13HZ2' for key 'PRIMARY'

01F9VG0NT0ACNSHJM30062WA6F
01F9VG0NT0ACNSHJM30297Z6DR
01F9VG0NT0ACNSHJM30062WA6F
Error 1062: Duplicate entry '01F9VG0NT0ACNSHJM30062WA6F' for key 'PRIMARY'

01F9VG0NT0ACNSHJM39T0D5B1B
01F9VG0NT0ACNSHJM39ZVD159R
01F9VG0NT0ACNSHJM3A59EMVZC
01F9VG0NT0ACNSHJM39XSN0A84
01F9VG0NT0ACNSHJM3A34FHCAR
01F9VG0NT0ACNSHJM3A5CKZKJT
01F9VG0NT0ACNSHJM39T0D5B1B
Error 1062: Duplicate entry '01F9VG0NT0ACNSHJM39T0D5B1B' for key 'PRIMARY'

01F9VG0NT0ACNSHJM3APRCBXA7
01F9VG0NT0ACNSHJM3ATGM0XHQ
01F9VG0NT0ACNSHJM3APRCBXA7
Error 1062: Duplicate entry '01F9VG0NT0ACNSHJM3APRCBXA7' for key 'PRIMARY'

01F9VG0NT0ACNSHJM3CTT1B09C
01F9VG0NT0ACNSHJM3CZHHK5QT
01F9VG0NT0ACNSHJM3D12FM3C4
01F9VG0NT0ACNSHJM3CXQGM6GM
01F9VG0NT0ACNSHJM3D3H1M9EE
01F9VG0NT0ACNSHJM3D4REME36
01F9VG0NT0ACNSHJM3CTT1B09C
Error 1062: Duplicate entry '01F9VG0NT0ACNSHJM3CTT1B09C' for key 'PRIMARY'

01F9VG0NT0ACNSHJM3DPF00J7M
01F9VG0NT0ACNSHJM3DG1C0KAC
01F9VG0NT0ACNSHJM3DQR5G89S
01F9VG0NT0ACNSHJM3DV3SHQ6M
01F9VG0NT0ACNSHJM3DY6049MW
01F9VG0NT0ACNSHJM3DPF00J7M
Error 1062: Duplicate entry '01F9VG0NT0ACNSHJM3DPF00J7M' for key 'PRIMARY'

01F9VG0NT0ACNSHJM3VYDT1FKJ
01F9VG0NT0ACNSHJM3VYDT1FKJ
Error 1062: Duplicate entry '01F9VG0NT0ACNSHJM3VYDT1FKJ' for key 'PRIMARY'

My code is broken into three parts, as it's a complex long running process:

stored in a struct:

now := time.Now().UTC()
entropy := ulid.Monotonic(rand.New(rand.NewSource(now.UnixNano())), 0)

I generate the ID (m is a struct):

job.ID = common.GenUlid(m.now, m.entropy)

Func:

func GenUlid(now time.Time, entropy io.Reader) string {
	return ulid.MustNew(ulid.Timestamp(now), entropy).String()
}

Thanks for your time.

I would like to change the 80 bit section to 48 bit, which part is it safe to change?

I would like to have a 12byte total length of ulid so instead of 80bit, i would like it to be 48 bit for the random id. where is it safe to change this? pls give your opinion. thank you.

1. type uint80 struct {
   Hi uint16
   Lo uint64 // <- should i change here? to uint32?
   }

2. or here

   const EncodedSize = 26 //<- if so, what should i change to?

[Question] Replace math.Rand with crypto.Reader in README maybe?

I've benchmarked ULID with different sources of entropy and found that with crypto.rand.Reader it works 10 times faster then math.Rand, at least on MacOS, it's very possible on linux it will behave different.

So maybe it's for better to change README to:

import "crypto/rand"

func ExampleULID() {
    t := time.Unix(1000000, 0)
    fmt.Println(ulid.MustNew(ulid.Timestamp(t), rand.Reader))
    // Output: 0000XSNJG0MQJHBF4QX1EFD6Y3
}

How do you think? I can submit benchmarks code if you will although those are trivial.

Feature/Generate ULID with lowercase characters

Is it possible to add a feature to have the final ULID to contain lowercase characters? My team is trying to use this package to generate unique folder names for Azure blob storage, and naming limitations says we cannot use upper case letters.

Incompatibility with Gorm

I'm having some trouble with getting this library playing together with Gorm and I'm not entirely sure where the fault is at. Maybe someone here could help point me in a direction to figure out where (and how) to solve this?

The problem is that Gorm doesn't seem to understand when ulid.ULIDs are empty, so when I have such a field it tried writing to it, which causes foreign key failure.

I've seen that this library implements a SQL Valuer and it looks good but, still, it doesn't work for me.

Any ideas what could be going wrong?

Here's an example:

package main

import (
	"math/rand"
	"os"
	"time"

	"github.com/oklog/ulid/v2"
	"gorm.io/driver/mysql"
	"gorm.io/gorm"
)

type One struct {
	ID    ulid.ULID `gorm:"primaryKey;type:varbinary(255)"`
	Name  string
	Two   *Two
	TwoID ulid.ULID
}

type Two struct {
	ID ulid.ULID `gorm:"primaryKey;type:varbinary(255)"`
}

func main() {
	db, _ := gorm.Open(mysql.Open(os.Getenv("DB_DSN")), &gorm.Config{})

	db.AutoMigrate(One{})
	db.AutoMigrate(Two{})

	now := time.Now()
	entropy := ulid.Monotonic(rand.New(rand.NewSource(now.UnixNano())), 0)

	one := One{ID: ulid.MustNew(ulid.Timestamp(now), entropy), Name: "Asdf"}

	db.Create(one)
}

Prior art comment

I notice that the README contains "Prior art". How about adding a sentence about the other Go alternatives explaining why this library exists?

Standalone executable

Are you publishing a standalone executable for the go version? Someone asked over on my repo, and I wanted to send them your way, because doing it with go would be pretty straightforward.

Support parsing UUID strings

Has there been any consideration into allowing string UUIDs be parsed within the unexported parse function: func parse(v []byte, strict bool, id *ULID) error

The reason I'm raising this is because I'd like to use this library for my primary keys, however CockroachDB returns UUID columns as UUID strings, not []byte, so ulid.ULID#Scan(src) fails to parse the value.

My workaround is to implement a wrapper ULID type which has a Scan function (implementation shown below) however it would be nice to not have to do this, as I have to do things like ULID{ULID: ulid.MustParse(...)} which is a real annoyance:

func (id *ULID) Scan(src interface{}) error {
	if str, ok := src.(string); ok {
		if len(str) != ulid.EncodedSize {
			bytes, err := uuid.Parse(str)
			if err != nil {
				return err
			}

			return id.UnmarshalBinary(bytes[:])
		}
	}

	return id.ULID.Scan(src)
}

If you don't want to implement UUID parsing or add an extra dependency, how about some mechanism where users can register custom parsers?

Issues generating ULID with freq. <1ms

Hi,

I've been experimenting with your library to generate ULIDs as keys for a database.
My code was doing 20+ row insertions per 1ms, and results, i.e. row ULIDs, are not in a strict sequence, not sorted (image below).
I was suggested that this may be due to "Within the same millisecond, sort order is not guaranteed" on https://github.com/alizain/ulid which, as I understand, served as a basis for oklog/ulid.

Could you please confirm (or not) randomness within 1 ms?
If not, below is my code, I would appreciate your check if IDs are generated correctly:

func GetNewID() string {
	entropy := rand.New(rand.NewSource(time.Now().UnixNano()))
	value := ulid.MustNew(ulid.Now(), entropy)
	return value.String()
}

This is a select in descending order by id (generated by ULID), where post_id column is filled with a "for" loop key. With correct generation, post_id should be a in a strict desc. order.

Thank you!
D.

I've changed to 12bytes for ulid, just curious if it's "ok"

removed

ulid.Monotonic isn't, when protected with a sync.Mutex

See #44 for a full description of the issue.

Use init() instead of sync.Once to initiate default entropy

In the DefaultEntropy you are using sync.Once to initiate the entropy and new reader. This means that every call to Make() has to make atomic call to see if sync.Once has been invoked. This seems very pointless when you could have simply used init(), which is run only once, and poses no performance hit. Especially when this is not ephemeral code but rather eternal and called many many many times.

How do you use this library?

func ExampleULID() {
	t := time.Unix(1000000, 0)
	entropy := ulid.Monotonic(rand.New(rand.NewSource(t.UnixNano())), 0)
	fmt.Println(ulid.MustNew(ulid.Timestamp(t), entropy))
	// Output: 0000XSNJG0MQJHBF4QX1EFD6Y3
}

Generates the same id?
think from a developer standpoint... what is the first thing they are looking for?
-> generating an id for a fixed time?

Convenience method to convert unix time to time.Time?

The package exposes a Timestamp(...) convenience function to convert from time.Time to a UNIX time in a uint64 but no method to go the other way. Was it a deliberate decision not to include this and, if so, would you mind explaining the rationale?

Thanks again for this awesome package!

Fix description of Make()

The Make() function states that LockedMonotonicReader uses sync.Pool but it actually uses sync.Mutex.