timtadh/fs2

B+ Tree - List - File Structures 2 - Memory Mapped File Structures for Go

fs2's Issues

Great work but documentation is restricted.

https://pkg.go.dev/github.com/timtadh/fs2/bptree?utm_source=godoc

The link is restricted. Possible to get it to work? Thx

to get range of item 100 - 1000, should i do like this or there's more efficient means to get 100-1000 items?

    //      var key, value []byte
    kvi, err := bpt.Find(k)
    if err != nil {  
            log.Fatal(err)
    }
    var i uint64
    var b [][]byte
    for key, value, err, kvi = kvi(); kvi != nil; key, value, err, kvi = kvi() {
            if i > 100 {
                     b[i] = value
            }
            if i == 1000 {
                  break;
            }
            i++

            // do stuff with the keys and values
            return value, err
    }
    if err != nil {
            log.Fatal(err)
    }
    return nil, err

Duplicate Key insert in internal block.

2015/06/25 14:32:52 would have inserted a duplicate key, [0 0 0 40]
goroutine 5 [running]:
github.com/timtadh/fs2/errors.Errorf(0xc20e8272c0, 0x2f, 0x0, 0x0, 0x0, 0x0, 0x0)
        /home/hendersont/stuff/research/fsm/src/github.com/timtadh/fs2/errors/error.go:16 +0x92
github.com/timtadh/fs2/bptree.(*internal).putKey(0x7fded5867000, 0x0, 0x7fded63de020, 0x4, 0xfe0, 0xc20ee88ea8, 0x0, 0x0)
        /home/hendersont/stuff/research/fsm/src/github.com/timtadh/fs2/bptree/internal.go:277 +0x3c4
github.com/timtadh/fs2/bptree.(*internal).putKP(0x7fded5867000, 0x0, 0x7fded63de020, 0x4, 0xfe0, 0xbe2000, 0x0, 0x0)
        /home/hendersont/stuff/research/fsm/src/github.com/timtadh/fs2/bptree/internal.go:211 +0x1d8
github.com/timtadh/fs2/bptree.func·073(0x7fded63de020, 0x4, 0xfe0, 0x0, 0x0)
        /home/hendersont/stuff/research/fsm/src/github.com/timtadh/fs2/bptree/insert.go:156 +0x18b
github.com/timtadh/fs2/bptree.func·023(0x7fded63de000, 0x0, 0x0)
        /home/hendersont/stuff/research/fsm/src/github.com/timtadh/fs2/bptree/first.go:20 +0x124
github.com/timtadh/fs2/bptree.func·021(0x7fded63de000, 0x1000, 0x1000, 0x0, 0x0)
        /home/hendersont/stuff/research/fsm/src/github.com/timtadh/fs2/bptree/do.go:103 +0xfa
github.com/timtadh/fs2/fmap.(*BlockFile).Do(0xc20cc28240, 0xbe2000, 0x1, 0xc20ee890a8, 0x0, 0x0)
        /home/hendersont/stuff/research/fsm/src/github.com/timtadh/fs2/fmap/fmap.go:559 +0xbf
github.com/timtadh/fs2/bptree.(*BpTree).do(0xc20fc78fc0, 0xbe2000, 0xc20ee89108, 0xc20ee890f8, 0x0, 0x0)
        /home/hendersont/stuff/research/fsm/src/github.com/timtadh/fs2/bptree/do.go:107 +0x79
github.com/timtadh/fs2/bptree.(*BpTree).firstKey(0xc20fc78fc0, 0xbe2000, 0xc20ee891b8, 0x0, 0x0)
        /home/hendersont/stuff/research/fsm/src/github.com/timtadh/fs2/bptree/first.go:22 +0x84
github.com/timtadh/fs2/bptree.func·074(0x7fded5867000, 0x0, 0x0)
        /home/hendersont/stuff/research/fsm/src/github.com/timtadh/fs2/bptree/insert.go:157 +0x253
github.com/timtadh/fs2/bptree.func·021(0x7fded5867000, 0x1000, 0x1000, 0x0, 0x0)
        /home/hendersont/stuff/research/fsm/src/github.com/timtadh/fs2/bptree/do.go:101 +0xab
github.com/timtadh/fs2/fmap.(*BlockFile).Do(0xc20cc28240, 0x6b000, 0x1, 0xc20ee892e0, 0x0, 0x0)
        /home/hendersont/stuff/research/fsm/src/github.com/timtadh/fs2/fmap/fmap.go:559 +0xbf
github.com/timtadh/fs2/bptree.(*BpTree).do(0xc20fc78fc0, 0x6b000, 0xc20ee89400, 0x685ea8, 0x0, 0x0)
        /home/hendersont/stuff/research/fsm/src/github.com/timtadh/fs2/bptree/do.go:107 +0x79
github.com/timtadh/fs2/bptree.(*BpTree).doInternal(0xc20fc78fc0, 0x6b000, 0xc20ee89400, 0x0, 0x0)
        /home/hendersont/stuff/research/fsm/src/github.com/timtadh/fs2/bptree/do.go:43 +0x5a
github.com/timtadh/fs2/bptree.(*BpTree).internalInsert(0xc20fc78fc0, 0x6b000, 0xc209b61dc0, 0x4, 0x4, 0xc209b61dc8, 0x8, 0x8, 0x18, 0xc209b61dc8, ...)
        /home/hendersont/stuff/research/fsm/src/github.com/timtadh/fs2/bptree/insert.go:160 +0x273
github.com/timtadh/fs2/bptree.(*BpTree).insert(0xc20fc78fc0, 0x6b000, 0xc209b61dc0, 0x4, 0x4, 0xc209b61dc8, 0x8, 0x8, 0x0, 0xc209b61dc8, ...)
        /home/hendersont/stuff/research/fsm/src/github.com/timtadh/fs2/bptree/insert.go:100 +0x13c
github.com/timtadh/fs2/bptree.(*BpTree).Add(0xc20fc78fc0, 0xc209b61dc0, 0x4, 0x4, 0xc209b61dc8, 0x8, 0x8, 0x0, 0x0)
        /home/hendersont/stuff/research/fsm/src/github.com/timtadh/fs2/bptree/insert.go:54 +0x2ff
github.com/timtadh/fs2/bptree.(*Varchar).szAdd(0xc20f688390, 0x24, 0xa24c34, 0x0, 0x0)
        /home/hendersont/stuff/research/fsm/src/github.com/timtadh/fs2/bptree/varchar.go:407 +0xd2
github.com/timtadh/fs2/bptree.(*Varchar).free(0xc20f688390, 0xa24c34, 0x24, 0x0, 0x0)
        /home/hendersont/stuff/research/fsm/src/github.com/timtadh/fs2/bptree/varchar.go:397 +0x5f2
github.com/timtadh/fs2/bptree.(*Varchar).freeExtra(0xc20f688390, 0xa24bc8, 0x90, 0x6c, 0x0, 0x0, 0x0)
        /home/hendersont/stuff/research/fsm/src/github.com/timtadh/fs2/bptree/varchar.go:268 +0xfd
github.com/timtadh/fs2/bptree.(*Varchar).newRun(0xc20f688390, 0xa24bc8, 0x5c, 0x6c, 0x90, 0x0, 0x0)
        /home/hendersont/stuff/research/fsm/src/github.com/timtadh/fs2/bptree/varchar.go:675 +0x13a
github.com/timtadh/fs2/bptree.(*Varchar).Alloc(0xc20f688390, 0x5c, 0xa24bc8, 0x0, 0x0)
        /home/hendersont/stuff/research/fsm/src/github.com/timtadh/fs2/bptree/varchar.go:222 +0x479
github.com/timtadh/fs2/bptree.(*BpTree).checkValue(0xc20fc78fe0, 0xc20e6f8f00, 0x5c, 0x5c, 0x0, 0x0, 0x0, 0x0, 0x0)
        /home/hendersont/stuff/research/fsm/src/github.com/timtadh/fs2/bptree/insert.go:15 +0xa8
github.com/timtadh/fs2/bptree.(*BpTree).Add(0xc20fc78fe0, 0xc20e6f8d20, 0x5c, 0x5c, 0xc20e6f8f00, 0x5c, 0x5c, 0x0, 0x0)
        /home/hendersont/stuff/research/fsm/src/github.com/timtadh/fs2/bptree/insert.go:50 +0x230
github.com/timtadh/fsm/store.(*Fs2BpTree).Add(0xc20fc79020, 0xc20e6f8d20, 0x5c, 0x5c, 0xc20bfc5f00)
        /home/hendersont/stuff/research/fsm/src/github.com/timtadh/fsm/store/fs.go:146 +0x769
github.com/timtadh/fsm/mine.(*Queue).Add(0xc20cc282c0, 0xc20ba58c00)
        /home/hendersont/stuff/research/fsm/src/github.com/timtadh/fsm/mine/queue.go:66 +0x1d1
github.com/timtadh/fsm/mine.func·014(0xc20ba58c00)
        /home/hendersont/stuff/research/fsm/src/github.com/timtadh/fsm/mine/depth.go:123 +0x79
github.com/timtadh/fsm/mine.(*DepthMiner).process(0xc20e6a41c0, 0xc20c0f1020, 0xc20ee89f18)
        /home/hendersont/stuff/research/fsm/src/github.com/timtadh/fsm/mine/depth.go:181 +0x440
github.com/timtadh/fsm/mine.(*DepthMiner).search(0xc20e6a41c0, 0xa)
        /home/hendersont/stuff/research/fsm/src/github.com/timtadh/fsm/mine/depth.go:128 +0x59f
github.com/timtadh/fsm/mine.(*DepthMiner).mine(0xc20e6a41c0)
        /home/hendersont/stuff/research/fsm/src/github.com/timtadh/fsm/mine/depth.go:79 +0x31
created by github.com/timtadh/fsm/mine.Depth
        /home/hendersont/stuff/research/fsm/src/github.com/timtadh/fsm/mine/depth.go:73 +0x321
Command exited with non-zero status 1

Internal double put

2015/11/29 22:19:15 <internal (debug) (vchar <nil>) meta: <{1 4 3 340}>, keys: <[[0 0 0 28] [0 0 11 92] [0 0 16 0]]>, ptrs: <[1024000 819200 819200]>>
2015/11/29 22:19:15 key [0 0 11 92] v was nil
2015/11/29 22:19:15 replacing [0 0 16 0]
2015/11/29 22:19:15 <internal (debug) (vchar <nil>) meta: <{1 4 3 340}>, keys: <[[0 0 0 28] [0 0 11 92] [0 0 16 0]]>, ptrs: <[1024000 819200 819200]>>
2015/11/29 22:19:15 was removing: [0 0 16 0]
2015/11/29 22:19:15 n: 831488, parent: 0, sibling: 0, okid 823296, kid: 819200
2015/11/29 22:19:15 internal already had key [0 0 11 92], at 1, was going to put it at 2, replacing [0 0 16 0]
goroutine 35 [running]:
github.com/timtadh/fs2/errors.Errorf(0x6d6fa0, 0x4b, 0xc8202f2d70, 0x4, 0x4, 0x0, 0x0)
        /home/hendersont/stuff/research/sfp/src/github.com/timtadh/fs2/errors/error.go:16 +0x89
github.com/timtadh/fs2/bptree.(*internal).updateK(0x7fd0be3a4000, 0x0, 0x2, 0x7fd0be3a1020, 0x4, 0xfe0, 0x0, 0x0)
        /home/hendersont/stuff/research/sfp/src/github.com/timtadh/fs2/bptree/internal.go:229 +0x5ff
github.com/timtadh/fs2/bptree.(*BpTree).internalDelete.func4.1(0x7fd0be3a1020, 0x4, 0xfe0, 0x0, 0x0)
        /home/hendersont/stuff/research/sfp/src/github.com/timtadh/fs2/bptree/remove.go:118 +0x63
github.com/timtadh/fs2/bptree.(*BpTree).firstKey.func2(0x7fd0be3a1000, 0x0, 0x0)
        /home/hendersont/stuff/research/sfp/src/github.com/timtadh/fs2/bptree/first.go:20 +0xed
github.com/timtadh/fs2/bptree.(*BpTree).do.func1(0x7fd0be3a1000, 0x1000, 0x1000, 0x0, 0x0)
        /home/hendersont/stuff/research/sfp/src/github.com/timtadh/fs2/bptree/do.go:103 +0xe7
github.com/timtadh/fs2/fmap.(*BlockFile).Do(0xc8200124c0, 0xc8000, 0x1, 0xc8202f2f80, 0x0, 0x0)
        /home/hendersont/stuff/research/sfp/src/github.com/timtadh/fs2/fmap/fmap.go:563 +0xb6
github.com/timtadh/fs2/bptree.(*BpTree).do(0xc8200108a0, 0xc8000, 0xc8202f2fe0, 0xc8202f2fd0, 0x0, 0x0)
        /home/hendersont/stuff/research/sfp/src/github.com/timtadh/fs2/bptree/do.go:107 +0x69
github.com/timtadh/fs2/bptree.(*BpTree).firstKey(0xc8200108a0, 0xc8000, 0xc8202f3020, 0x0, 0x0)
        /home/hendersont/stuff/research/sfp/src/github.com/timtadh/fs2/bptree/first.go:22 +0x6f
github.com/timtadh/fs2/bptree.(*BpTree).internalDelete.func4(0x7fd0be3a4000, 0x0, 0x0)
        /home/hendersont/stuff/research/sfp/src/github.com/timtadh/fs2/bptree/remove.go:119 +0x93
github.com/timtadh/fs2/bptree.(*BpTree).do.func1(0x7fd0be3a4000, 0x1000, 0x1000, 0x0, 0x0)
        /home/hendersont/stuff/research/sfp/src/github.com/timtadh/fs2/bptree/do.go:101 +0x98
github.com/timtadh/fs2/fmap.(*BlockFile).Do(0xc8200124c0, 0xcb000, 0x1, 0xc8202f3170, 0x0, 0x0)
        /home/hendersont/stuff/research/sfp/src/github.com/timtadh/fs2/fmap/fmap.go:563 +0xb6
github.com/timtadh/fs2/bptree.(*BpTree).do(0xc8200108a0, 0xcb000, 0xc8202f3300, 0x6ec470, 0x0, 0x0)
        /home/hendersont/stuff/research/sfp/src/github.com/timtadh/fs2/bptree/do.go:107 +0x69
github.com/timtadh/fs2/bptree.(*BpTree).doInternal(0xc8200108a0, 0xcb000, 0xc8202f3300, 0x0, 0x0)
        /home/hendersont/stuff/research/sfp/src/github.com/timtadh/fs2/bptree/do.go:43 +0x4d
github.com/timtadh/fs2/bptree.(*BpTree).internalDelete(0xc8200108a0, 0x0, 0xcb000, 0x0, 0xc8203a2000, 0x4, 0x4, 0xc8202f3530, 0x7fd0c15d7970, 0x0, ...)
        /home/hendersont/stuff/research/sfp/src/github.com/timtadh/fs2/bptree/remove.go:120 +0x894
github.com/timtadh/fs2/bptree.(*BpTree).delete(0xc8200108a0, 0x0, 0xcb000, 0x0, 0xc8203a2000, 0x4, 0x4, 0xc8202f3530, 0xc8203a2000, 0x0, ...)
        /home/hendersont/stuff/research/sfp/src/github.com/timtadh/fs2/bptree/remove.go:58 +0x120
github.com/timtadh/fs2/bptree.(*BpTree).remove(0xc8200108a0, 0xcb000, 0xc8203a2000, 0x4, 0x4, 0xc8202f3530, 0x4, 0x4, 0x0, 0x0)
        /home/hendersont/stuff/research/sfp/src/github.com/timtadh/fs2/bptree/remove.go:35 +0x83
github.com/timtadh/fs2/bptree.(*BpTree).Remove(0xc8200108a0, 0xc8203a2000, 0x4, 0x4, 0xc8202f3530, 0x0, 0x0)
        /home/hendersont/stuff/research/sfp/src/github.com/timtadh/fs2/bptree/remove.go:25 +0x6d
github.com/timtadh/fs2/bptree.(*Varchar).szDel(0xc82001a720, 0x1000, 0xcc000, 0x0, 0x0)
        /home/hendersont/stuff/research/sfp/src/github.com/timtadh/fs2/bptree/varchar.go:413 +0x2e3
github.com/timtadh/fs2/bptree.(*Varchar).indexRemove(0xc82001a720, 0x1000, 0xcc000, 0x0, 0x0)
        /home/hendersont/stuff/research/sfp/src/github.com/timtadh/fs2/bptree/varchar.go:425 +0x5c
github.com/timtadh/fs2/bptree.(*Varchar).Alloc(0xc82001a720, 0x68, 0xcc000, 0x0, 0x0)
        /home/hendersont/stuff/research/sfp/src/github.com/timtadh/fs2/bptree/varchar.go:220 +0x52e
github.com/timtadh/fs2/bptree.(*Varchar).allocNew(0xc82001a720, 0x68, 0x78, 0x0, 0x0, 0x0)
        /home/hendersont/stuff/research/sfp/src/github.com/timtadh/fs2/bptree/varchar.go:245 +0x12a
github.com/timtadh/fs2/bptree.(*Varchar).Alloc(0xc82001a720, 0x68, 0x0, 0x0, 0x0)
        /home/hendersont/stuff/research/sfp/src/github.com/timtadh/fs2/bptree/varchar.go:218 +0x4d7
github.com/timtadh/fs2/bptree.(*BpTree).checkValue(0xc8200108c0, 0xc82065b1f0, 0x68, 0x68, 0x0, 0x0, 0x0, 0x0, 0x0)
        /home/hendersont/stuff/research/sfp/src/github.com/timtadh/fs2/bptree/insert.go:16 +0x9e
github.com/timtadh/fs2/bptree.(*BpTree).Add(0xc8200108c0, 0xc8200c6120, 0x5c, 0x5c, 0xc82065b1f0, 0x68, 0x68, 0x0, 0x0)
        /home/hendersont/stuff/research/sfp/src/github.com/timtadh/fs2/bptree/insert.go:51 +0x254
github.com/timtadh/sfp/types/graph.(*Node).cache(0xc820613a80, 0x7fd0bfde3438, 0xc8200109a0, 0x7fd0bfde33b0, 0xc8200108c0, 0xc8200c6120, 0x5c, 0x5c, 0xc8201b7240, 0x3, ...)
        /home/hendersont/stuff/research/sfp/src/github.com/timtadh/sfp/types/graph/node.go:304 +0x299
github.com/timtadh/sfp/types/graph.(*Node).Children(0xc820613a80, 0xc8201b7240, 0x3, 0x4, 0x0, 0x0)
        /home/hendersont/stuff/research/sfp/src/github.com/timtadh/sfp/types/graph/node.go:286 +0xa55
github.com/timtadh/sfp/miners/absorbing.walk(0xc820012580, 0x0, 0x0, 0x0, 0x0)
        /home/hendersont/stuff/research/sfp/src/github.com/timtadh/sfp/miners/absorbing/mine.go:176 +0x166
github.com/timtadh/sfp/miners/absorbing.RejectingWalk.func1(0xc820012580, 0xc820258120, 0xc8202580c0, 0xc820258060)
        /home/hendersont/stuff/research/sfp/src/github.com/timtadh/sfp/miners/absorbing/mine.go:149 +0x30
created by github.com/timtadh/sfp/miners/absorbing.RejectingWalk
        /home/hendersont/stuff/research/sfp/src/github.com/timtadh/sfp/miners/absorbing/mine.go:163 +0xbb

goroutine 1 [runnable]:
github.com/timtadh/sfp/stores/bytes_subgraph.(*BpTree).kvIter.func1(0xc8203fc4c0, 0xc, 0xc, 0xc8206d9400, 0x0, 0x0, 0xc820368480)
        /home/hendersont/stuff/research/sfp/src/github.com/timtadh/sfp/stores/bytes_subgraph/wrapper.go:254
github.com/timtadh/sfp/stores/bytes_subgraph.Do(0xc8202f6dd8, 0xc8203fc060, 0x0, 0x0)
        /home/hendersont/stuff/research/sfp/src/github.com/timtadh/sfp/stores/bytes_subgraph/wrapper.go:88 +0x129
github.com/timtadh/sfp/stores/bytes_subgraph.(*BpTree).DoFind(0xc8201320c0, 0xc8203fc050, 0xc, 0xc, 0xc8203fc060, 0x0, 0x0)
        /home/hendersont/stuff/research/sfp/src/github.com/timtadh/sfp/stores/bytes_subgraph/wrapper.go:339 +0x68
github.com/timtadh/sfp/types/graph.edgeChain(0xc820096000, 0xc8206be300, 0x5c, 0xc8206f1ef0, 0x5, 0x5, 0x0, 0x0)
        /home/hendersont/stuff/research/sfp/src/github.com/timtadh/sfp/types/graph/node.go:212 +0x359
github.com/timtadh/sfp/types/graph.(*Node).FindNode(0xc82056d680, 0xc820096000, 0xc8206be300, 0x0, 0x0, 0x0)
        /home/hendersont/stuff/research/sfp/src/github.com/timtadh/sfp/types/graph/node.go:159 +0x3b3
github.com/timtadh/sfp/types/graph.(*Node).Parents(0xc82056d680, 0x0, 0x0, 0x0, 0x0, 0x0)
        /home/hendersont/stuff/research/sfp/src/github.com/timtadh/sfp/types/graph/node.go:128 +0x50d
github.com/timtadh/sfp/lattice.lattice(0x7fd0be09b838, 0xc82056d680, 0x7fd0bfdf5738, 0x0, 0x0)
        /home/hendersont/stuff/research/sfp/src/github.com/timtadh/sfp/lattice/lattice.go:35 +0x359
github.com/timtadh/sfp/lattice.MakeLattice(0x7fd0be09b838, 0xc82056d680, 0xb00, 0x0, 0x0)
        /home/hendersont/stuff/research/sfp/src/github.com/timtadh/sfp/lattice/lattice.go:16 +0xaf
github.com/timtadh/sfp/miners/absorbing.(*Walker).PrMatrices(0xc820012580, 0x7fd0be09b838, 0xc82056d680, 0xc820464440, 0xc8204c8c00, 0xa54, 0x0, 0x0)
        /home/hendersont/stuff/research/sfp/src/github.com/timtadh/sfp/miners/absorbing/mine.go:37 +0x54
github.com/timtadh/sfp/miners/absorbing.(*MatrixReporter).Report(0xc820225980, 0x7fd0be09b838, 0xc82056d680, 0x0, 0x0)
        /home/hendersont/stuff/research/sfp/src/github.com/timtadh/sfp/miners/absorbing/reporter.go:43 +0x62
github.com/timtadh/sfp/miners/reporters.(*Chain).Report(0xc820164800, 0x7fd0be09b838, 0xc82056d680, 0x0, 0x0)
        /home/hendersont/stuff/research/sfp/src/github.com/timtadh/sfp/miners/reporters/chain.go:18 +0xcb
github.com/timtadh/sfp/miners/walker.(*Walker).Mine(0xc820012580, 0x7fd0be09bc68, 0xc820096000, 0x7fd0be09bd80, 0xc820164800, 0x0, 0x0)
        /home/hendersont/stuff/research/sfp/src/github.com/timtadh/sfp/miners/walker/walker.go:58 +0x219
github.com/timtadh/sfp/miners/absorbing.(*Walker).Mine(0xc820012580, 0x7fd0be09bc68, 0xc820096000, 0x7fd0be09bd80, 0xc820164740, 0x0, 0x0)
        /home/hendersont/stuff/research/sfp/src/github.com/timtadh/sfp/miners/absorbing/mine.go:140 +0x1bf
main.main()
        /home/hendersont/stuff/research/sfp/src/github.com/timtadh/sfp/main.go:556 +0x1737

goroutine 17 [syscall, locked to thread]:
runtime.goexit()
        /home/hendersont/srcs/go/src/runtime/asm_amd64.s:1696 +0x1

goroutine 36 [chan receive]:
github.com/timtadh/sfp/miners/walker.(*Walker).RejectingWalk.func1(0xc820258060, 0xc820012580, 0xc820258180, 0xc8202580c0)
        /home/hendersont/stuff/research/sfp/src/github.com/timtadh/sfp/miners/walker/walker.go:75 +0x5d
created by github.com/timtadh/sfp/miners/walker.(*Walker).RejectingWalk
        /home/hendersont/stuff/research/sfp/src/github.com/timtadh/sfp/miners/walker/walker.go:89 +0x71

There was error during the mining process
Unknown block type
goroutine 1 [running]:
github.com/timtadh/fs2/errors.Errorf(0x6a9b00, 0x12, 0x0, 0x0, 0x0, 0x0, 0x0)
        /home/hendersont/stuff/research/sfp/src/github.com/timtadh/fs2/errors/error.go:16 +0x89
github.com/timtadh/fs2/bptree.(*BpTree).insert(0xc8200108a0, 0xc8000, 0xc82044b980, 0x4, 0x4, 0xc82044b990, 0x8, 0x8, 0x1, 0xc8202f6070, ...)
        /home/hendersont/stuff/research/sfp/src/github.com/timtadh/fs2/bptree/insert.go:110 +0x226
github.com/timtadh/fs2/bptree.(*BpTree).internalInsert(0xc8200108a0, 0xcb000, 0xc82044b980, 0x4, 0x4, 0xc82044b990, 0x8, 0x8, 0x1, 0x0, ...)
        /home/hendersont/stuff/research/sfp/src/github.com/timtadh/fs2/bptree/insert.go:141 +0x172
github.com/timtadh/fs2/bptree.(*BpTree).insert(0xc8200108a0, 0xcb000, 0xc82044b980, 0x4, 0x4, 0xc82044b990, 0x8, 0x8, 0xc82044b901, 0x0, ...)
        /home/hendersont/stuff/research/sfp/src/github.com/timtadh/fs2/bptree/insert.go:106 +0x144
github.com/timtadh/fs2/bptree.(*BpTree).add(0xc8200108a0, 0xcb000, 0xc82044b980, 0x4, 0x4, 0xc82044b990, 0x8, 0x8, 0x1, 0xc82044b990, ...)
        /home/hendersont/stuff/research/sfp/src/github.com/timtadh/fs2/bptree/insert.go:65 +0xab
github.com/timtadh/fs2/bptree.(*BpTree).Add(0xc8200108a0, 0xc82044b980, 0x4, 0x4, 0xc82044b990, 0x8, 0x8, 0x0, 0x0)
        /home/hendersont/stuff/research/sfp/src/github.com/timtadh/fs2/bptree/insert.go:55 +0x2fb
github.com/timtadh/fs2/bptree.(*Varchar).szAdd(0xc82001a720, 0x1000, 0xc8000, 0x0, 0x0)
        /home/hendersont/stuff/research/sfp/src/github.com/timtadh/fs2/bptree/varchar.go:401 +0x2bf
github.com/timtadh/fs2/bptree.(*Varchar).indexAdd(0xc82001a720, 0x1000, 0xc8000, 0x0, 0x0)
        /home/hendersont/stuff/research/sfp/src/github.com/timtadh/fs2/bptree/varchar.go:417 +0x5c
github.com/timtadh/fs2/bptree.(*Varchar).free(0xc82001a720, 0xc8000, 0x1000, 0x0, 0x0)
        /home/hendersont/stuff/research/sfp/src/github.com/timtadh/fs2/bptree/varchar.go:395 +0x734
github.com/timtadh/fs2/bptree.(*Varchar).allocNew(0xc82001a720, 0x2c, 0x3c, 0x0, 0x0, 0x0)
        /home/hendersont/stuff/research/sfp/src/github.com/timtadh/fs2/bptree/varchar.go:241 +0xea
github.com/timtadh/fs2/bptree.(*Varchar).Alloc(0xc82001a720, 0x2c, 0x0, 0x0, 0x0)
        /home/hendersont/stuff/research/sfp/src/github.com/timtadh/fs2/bptree/varchar.go:218 +0x4d7
github.com/timtadh/fs2/bptree.(*BpTree).newVarcharKey(0xc8200108c0, 0xce000, 0xc8205c27b0, 0x2c, 0x2c, 0x7fd0be3a7001, 0x0, 0x0, 0x0, 0x0, ...)
        /home/hendersont/stuff/research/sfp/src/github.com/timtadh/fs2/bptree/insert.go:210 +0x19e
github.com/timtadh/fs2/bptree.(*BpTree).leafInsert(0xc8200108c0, 0xce000, 0xc8205c27b0, 0x2c, 0x2c, 0xc82044b850, 0x8, 0x8, 0x6ec401, 0x5226dd, ...)
        /home/hendersont/stuff/research/sfp/src/github.com/timtadh/fs2/bptree/insert.go:227 +0xb9
github.com/timtadh/fs2/bptree.(*BpTree).insert(0xc8200108c0, 0xce000, 0xc8205c27b0, 0x2c, 0x2c, 0xc82044b850, 0x8, 0x8, 0x545a01, 0x7fd0be3a3fa4, ...)
        /home/hendersont/stuff/research/sfp/src/github.com/timtadh/fs2/bptree/insert.go:108 +0x1c0
github.com/timtadh/fs2/bptree.(*BpTree).internalInsert(0xc8200108c0, 0xf6000, 0xc8205c27b0, 0x2c, 0x2c, 0xc82044b850, 0x8, 0x8, 0x1, 0x0, ...)
        /home/hendersont/stuff/research/sfp/src/github.com/timtadh/fs2/bptree/insert.go:141 +0x172
github.com/timtadh/fs2/bptree.(*BpTree).insert(0xc8200108c0, 0xf6000, 0xc8205c27b0, 0x2c, 0x2c, 0xc82044b850, 0x8, 0x8, 0x1, 0x3c, ...)
        /home/hendersont/stuff/research/sfp/src/github.com/timtadh/fs2/bptree/insert.go:106 +0x144
github.com/timtadh/fs2/bptree.(*BpTree).add(0xc8200108c0, 0xf6000, 0xc8205c27b0, 0x2c, 0x2c, 0xc82044b850, 0x8, 0x8, 0x1, 0x0, ...)
        /home/hendersont/stuff/research/sfp/src/github.com/timtadh/fs2/bptree/insert.go:65 +0xab
github.com/timtadh/fs2/bptree.(*BpTree).Add(0xc8200108c0, 0xc8205c27b0, 0x2c, 0x2c, 0xc82044b850, 0x8, 0x8, 0x0, 0x0)
        /home/hendersont/stuff/research/sfp/src/github.com/timtadh/fs2/bptree/insert.go:55 +0x2fb
github.com/timtadh/sfp/types/graph.(*Node).cache(0xc8203a81c0, 0x7fd0bfde3438, 0xc8200109a0, 0x7fd0bfde33b0, 0xc8200108c0, 0xc8205c27b0, 0x2c, 0x2c, 0xc8201e3540, 0x2, ...)
        /home/hendersont/stuff/research/sfp/src/github.com/timtadh/sfp/types/graph/node.go:304 +0x299
github.com/timtadh/sfp/types/graph.(*Node).Children(0xc8203a81c0, 0xc8201e3540, 0x2, 0x2, 0x0, 0x0)
        /home/hendersont/stuff/research/sfp/src/github.com/timtadh/sfp/types/graph/node.go:286 +0xa55
github.com/timtadh/sfp/lattice.lattice(0x7fd0be09b838, 0xc82056d680, 0x7fd0bfdf5738, 0x0, 0x0)
        /home/hendersont/stuff/research/sfp/src/github.com/timtadh/sfp/lattice/lattice.go:55 +0x9fd
github.com/timtadh/sfp/lattice.MakeLattice(0x7fd0be09b838, 0xc82056d680, 0xb00, 0x0, 0x0)
        /home/hendersont/stuff/research/sfp/src/github.com/timtadh/sfp/lattice/lattice.go:16 +0xaf
github.com/timtadh/sfp/miners/absorbing.(*Walker).PrMatrices(0xc820012580, 0x7fd0be09b838, 0xc82056d680, 0xc820464440, 0xc8204c8c00, 0xa54, 0x0, 0x0)
        /home/hendersont/stuff/research/sfp/src/github.com/timtadh/sfp/miners/absorbing/mine.go:37 +0x54
github.com/timtadh/sfp/miners/absorbing.(*MatrixReporter).Report(0xc820225980, 0x7fd0be09b838, 0xc82056d680, 0x0, 0x0)
        /home/hendersont/stuff/research/sfp/src/github.com/timtadh/sfp/miners/absorbing/reporter.go:43 +0x62
github.com/timtadh/sfp/miners/reporters.(*Chain).Report(0xc820164800, 0x7fd0be09b838, 0xc82056d680, 0x0, 0x0)
        /home/hendersont/stuff/research/sfp/src/github.com/timtadh/sfp/miners/reporters/chain.go:18 +0xcb
github.com/timtadh/sfp/miners/walker.(*Walker).Mine(0xc820012580, 0x7fd0be09bc68, 0xc820096000, 0x7fd0be09bd80, 0xc820164800, 0x0, 0x0)
        /home/hendersont/stuff/research/sfp/src/github.com/timtadh/sfp/miners/walker/walker.go:58 +0x219
github.com/timtadh/sfp/miners/absorbing.(*Walker).Mine(0xc820012580, 0x7fd0be09bc68, 0xc820096000, 0x7fd0be09bd80, 0xc820164740, 0x0, 0x0)
        /home/hendersont/stuff/research/sfp/src/github.com/timtadh/sfp/miners/absorbing/mine.go:140 +0x1bf
main.main()
        /home/hendersont/stuff/research/sfp/src/github.com/timtadh/sfp/main.go:556 +0x1737

goroutine 17 [syscall, locked to thread]:
runtime.goexit()
        /home/hendersont/srcs/go/src/runtime/asm_amd64.s:1696 +0x1

goroutine 35 [chan send]:
github.com/timtadh/sfp/miners/absorbing.RejectingWalk.func1(0xc820012580, 0xc820258120, 0xc8202580c0, 0xc820258060)
        /home/hendersont/stuff/research/sfp/src/github.com/timtadh/sfp/miners/absorbing/mine.go:151 +0x8f
created by github.com/timtadh/sfp/miners/absorbing.RejectingWalk
        /home/hendersont/stuff/research/sfp/src/github.com/timtadh/sfp/miners/absorbing/mine.go:163 +0xbb

goroutine 36 [chan receive]:
github.com/timtadh/sfp/miners/walker.(*Walker).RejectingWalk.func1(0xc820258060, 0xc820012580, 0xc820258180, 0xc8202580c0)
        /home/hendersont/stuff/research/sfp/src/github.com/timtadh/sfp/miners/walker/walker.go:75 +0x5d
created by github.com/timtadh/sfp/miners/walker.(*Walker).RejectingWalk
        /home/hendersont/stuff/research/sfp/src/github.com/timtadh/sfp/miners/walker/walker.go:89 +0x71

Command exited with non-zero status 1

The 'count' of idxBlk is too small!

type idxBlk struct {
flags consts.Flag
count uint16
items [itemsPerIdx]uint64
}

uint32 is better.

MacOS incompatibility

It seems the package has incompatibility because of missing mremap on darwin platform.

I would appreciate if you can look into it? There are multiple options:

• Maybe mremap can be replaced with munmap followed by mmap with new size?
• Use mremap (and related mmap functions) available in newer version of GO https://pkg.go.dev/golang.org/x/[email protected]/unix#Mremap

Unexpected Free Blk

edit 2015-06-12 : This seems to be a very rare error. I have only seen it this once. Not exactly sure what caused it, it looks like either an error in leafInsert returning the wrong value of a key or perhaps a dangling pointer. Not sure how that would have happened since the program using this was never deleting items from the tree. I will keep my eye out for it.

2015/06/10 14:31:59 unexpected free blk
goroutine 183064 [running]:
github.com/timtadh/fs2/errors.Errorf(0x614490, 0x13, 0x0, 0x0, 0x0, 0x0, 0x0)
        /home/hendersont/stuff/research/fsm/src/github.com/timtadh/fs2/errors/error.go:16 +0x92
github.com/timtadh/fs2/bptree.func·144(0x7f03f021be60, 0x0, 0x0)
        /home/hendersont/stuff/research/fsm/src/github.com/timtadh/fs2/bptree/varchar.go:558 +0x62
github.com/timtadh/fs2/bptree.func·149(0x7f03f021be60, 0x1a0, 0x1a0, 0x0, 0x0)
        /home/hendersont/stuff/research/fsm/src/github.com/timtadh/fs2/bptree/varchar.go:605 +0x13d
github.com/timtadh/fs2/fmap.(*BlockFile).Do(0xc20d9214c0, 0xbb1d000, 0x1, 0xc20dc653f0, 0x0, 0x0)
        /home/hendersont/stuff/research/fsm/src/github.com/timtadh/fs2/fmap/fmap.go:558 +0xbf
github.com/timtadh/fs2/bptree.(*Varchar).do(0xc20f280f60, 0xbb1de60, 0x64a340, 0x64a348, 0x64a338, 0x0, 0x0)
        /home/hendersont/stuff/research/fsm/src/github.com/timtadh/fs2/bptree/varchar.go:611 +0xc1
github.com/timtadh/fs2/bptree.(*Varchar).doRun(0xc20f280f60, 0xbb1de60, 0x64a338, 0x0, 0x0)
        /home/hendersont/stuff/research/fsm/src/github.com/timtadh/fs2/bptree/varchar.go:560 +0x66
github.com/timtadh/fs2/bptree.(*Varchar).Ref(0xc20f280f60, 0xbb1de60, 0x0, 0x0)
        /home/hendersont/stuff/research/fsm/src/github.com/timtadh/fs2/bptree/varchar.go:507 +0x50
github.com/timtadh/fs2/bptree.(*internal).bigUpdateK(0x7f03e470f000, 0xc20f280f60, 0x0, 0x7f03e47fe020, 0x8, 0xfe0, 0x0, 0x0)
        /home/hendersont/stuff/research/fsm/src/github.com/timtadh/fs2/bptree/internal.go:181 +0x147
github.com/timtadh/fs2/bptree.(*internal).updateK(0x7f03e470f000, 0xc20f280f60, 0x0, 0x7f03e47fe020, 0x8, 0xfe0, 0x0, 0x0)
        /home/hendersont/stuff/research/fsm/src/github.com/timtadh/fs2/bptree/internal.go:168 +0x115
github.com/timtadh/fs2/bptree.func·072(0x7f03e47fe020, 0x8, 0xfe0, 0x0, 0x0)
        /home/hendersont/stuff/research/fsm/src/github.com/timtadh/fs2/bptree/insert.go:143 +0x77
github.com/timtadh/fs2/bptree.func·023(0x7f03e47fe000, 0x0, 0x0)
        /home/hendersont/stuff/research/fsm/src/github.com/timtadh/fs2/bptree/first.go:20 +0x124
github.com/timtadh/fs2/bptree.func·021(0x7f03e47fe000, 0x1000, 0x1000, 0x0, 0x0)
        /home/hendersont/stuff/research/fsm/src/github.com/timtadh/fs2/bptree/do.go:103 +0xdd
github.com/timtadh/fs2/fmap.(*BlockFile).Do(0xc20d9214c0, 0x100000, 0x1, 0xc20dc65750, 0x0, 0x0)
        /home/hendersont/stuff/research/fsm/src/github.com/timtadh/fs2/fmap/fmap.go:558 +0xbf
github.com/timtadh/fs2/bptree.(*BpTree).do(0xc20ff232e0, 0x100000, 0xc20dc657b0, 0xc20dc657a0, 0x0, 0x0)
        /home/hendersont/stuff/research/fsm/src/github.com/timtadh/fs2/bptree/do.go:107 +0x79
github.com/timtadh/fs2/bptree.(*BpTree).firstKey(0xc20ff232e0, 0x100000, 0xc20dc65840, 0x0, 0x0)
        /home/hendersont/stuff/research/fsm/src/github.com/timtadh/fs2/bptree/first.go:22 +0x84
github.com/timtadh/fs2/bptree.func·074(0x7f03e470f000, 0x0, 0x0)
        /home/hendersont/stuff/research/fsm/src/github.com/timtadh/fs2/bptree/insert.go:144 +0x12d
github.com/timtadh/fs2/bptree.func·021(0x7f03e470f000, 0x1000, 0x1000, 0x0, 0x0)
        /home/hendersont/stuff/research/fsm/src/github.com/timtadh/fs2/bptree/do.go:101 +0x8f
github.com/timtadh/fs2/fmap.(*BlockFile).Do(0xc20d9214c0, 0x11000, 0x1, 0xc20dc65988, 0x0, 0x0)
        /home/hendersont/stuff/research/fsm/src/github.com/timtadh/fs2/fmap/fmap.go:558 +0xbf
github.com/timtadh/fs2/bptree.(*BpTree).do(0xc20ff232e0, 0x11000, 0xc20dc65aa8, 0x64a318, 0x0, 0x0)
        /home/hendersont/stuff/research/fsm/src/github.com/timtadh/fs2/bptree/do.go:107 +0x79
github.com/timtadh/fs2/bptree.(*BpTree).doInternal(0xc20ff232e0, 0x11000, 0xc20dc65aa8, 0x0, 0x0)
        /home/hendersont/stuff/research/fsm/src/github.com/timtadh/fs2/bptree/do.go:43 +0x5a
github.com/timtadh/fs2/bptree.(*BpTree).internalInsert(0xc20ff232e0, 0x11000, 0xc20c11c820, 0x18c, 0x18c, 0xc209de2f40, 0x1, 0x1, 0x49fc3a, 0xc20ff232e0, ...)
        /home/hendersont/stuff/research/fsm/src/github.com/timtadh/fs2/bptree/insert.go:160 +0x273
github.com/timtadh/fs2/bptree.(*BpTree).insert(0xc20ff232e0, 0x11000, 0xc20c11c820, 0x18c, 0x18c, 0xc209de2f40, 0x1, 0x1, 0xc20dc65be0, 0x0, ...)
        /home/hendersont/stuff/research/fsm/src/github.com/timtadh/fs2/bptree/insert.go:100 +0x138
github.com/timtadh/fs2/bptree.(*BpTree).internalInsert(0xc20ff232e0, 0x7da2000, 0xc20c11c820, 0x18c, 0x18c, 0xc209de2f40, 0x1, 0x1, 0x49fec9, 0xc20d9214c0, ...)
        /home/hendersont/stuff/research/fsm/src/github.com/timtadh/fs2/bptree/insert.go:134 +0x17a
github.com/timtadh/fs2/bptree.(*BpTree).insert(0xc20ff232e0, 0x7da2000, 0xc20c11c820, 0x18c, 0x18c, 0xc209de2f40, 0x1, 0x1, 0x0, 0x0, ...)
        /home/hendersont/stuff/research/fsm/src/github.com/timtadh/fs2/bptree/insert.go:100 +0x138
github.com/timtadh/fs2/bptree.(*BpTree).Add(0xc20ff232e0, 0xc20c11c820, 0x18c, 0x18c, 0xc209de2f40, 0x1, 0x1, 0x0, 0x0)
        /home/hendersont/stuff/research/fsm/src/github.com/timtadh/fs2/bptree/insert.go:54 +0x2fd
github.com/timtadh/fsm/mine.addToLabels(0xc20ff232e0, 0xc20c11c820, 0x18c, 0x18c)
        /home/hendersont/stuff/research/fsm/src/github.com/timtadh/fsm/mine/maximal.go:65 +0x1c9
github.com/timtadh/fsm/mine.func·024()
        /home/hendersont/stuff/research/fsm/src/github.com/timtadh/fsm/mine/maximal.go:46 +0x3e5
created by github.com/timtadh/fsm/mine.MaximalSubGraphs
        /home/hendersont/stuff/research/fsm/src/github.com/timtadh/fsm/mine/maximal.go:55 +0x345

Possible to simulate a db.Batch() function like bbolt?

Possible to simulate a db.Batch() function like bbolt db?
because Add is a bit slow.

go.mod support

Any chance you could support go.mod?

Double Internal put

Not sure if this is a real issue or a usage error

2015/12/03 20:56:41 <internal (debug) (vchar <nil>) meta: <{1 4 39 340}>, keys: <[[0 0 0 24] [0 0 0 28] [0 0 0 32] [0 0 0 36] [0 0 0 40] [0 0 0 44] [0 0 0 48] [0 0 0 52] [0 0 0 56] [0 0 0 60] [0 0 0 64] [0 0 0 68] [0 0 0 72] [0 0 0 76] [0 0 0 80] [0 0 0 84] [0 0 0 88] [0 0 0 92] [0 0 0 96] [0 0 0 100] [0 0 0 104] [0 0 0 108] [0 0 0 112] [0 0 0 116] [0 0 0 120] [0 0 0 124] [0 0 0 128] [0 0 0 132] [0 0 0 136] [0 0 0 140] [0 0 0 144] [0 0 0 148] [0 0 0 152] [0 0 0 156] [0 0 0 164] [0 0 0 168] [0 0 0 176] [0 0 0 180] [0 0 0 208]]>, ptrs: <[1024000 765952 4583424 4440064 65822720 49954816 19202048 56877056 49369088 91881472 13000704 27590656 50708480 8515584 26238976 20332544 36560896 32595968 14405632 29827072 27242496 111857664 65892352 107053056 41676800 91000832 57516032 108855296 43786240 99532800 108916736 214274048 215560192 165625856 338407424 209829888 349868032 349970432 349982720]>>
2015/12/03 20:56:41 key [0 0 0 168] v was nil
2015/12/03 20:56:41 replacing [0 0 0 164]
2015/12/03 20:56:41 <internal (debug) (vchar <nil>) meta: <{1 4 39 340}>, keys: <[[0 0 0 24] [0 0 0 28] [0 0 0 32] [0 0 0 36] [0 0 0 40] [0 0 0 44] [0 0 0 48] [0 0 0 52] [0 0 0 56] [0 0 0 60] [0 0 0 64] [0 0 0 68] [0 0 0 72] [0 0 0 76] [0 0 0 80] [0 0 0 84] [0 0 0 88] [0 0 0 92] [0 0 0 96] [0 0 0 100] [0 0 0 104] [0 0 0 108] [0 0 0 112] [0 0 0 116] [0 0 0 120] [0 0 0 124] [0 0 0 128] [0 0 0 132] [0 0 0 136] [0 0 0 140] [0 0 0 144] [0 0 0 148] [0 0 0 152] [0 0 0 156] [0 0 0 164] [0 0 0 168] [0 0 0 176] [0 0 0 180] [0 0 0 208]]>, ptrs: <[1024000 765952 4583424 4440064 65822720 49954816 19202048 56877056 49369088 91881472 13000704 27590656 50708480 8515584 26238976 20332544 36560896 32595968 14405632 29827072 27242496 111857664 65892352 107053056 41676800 91000832 57516032 108855296 43786240 99532800 108916736 214274048 215560192 165625856 338407424 209829888 349868032 349970432 349982720]>>
2015/12/03 20:56:41 was removing: [0 0 0 164]
2015/12/03 20:56:41 n: 921600, parent: 0, sibling: 209829888, okid 338407424, kid: 338407424
2015/12/03 20:56:41 internal already had key [0 0 0 168], at 35, was going to put it at 34, replacing [0 0 0 164]
goroutine 34 [running]:
github.com/timtadh/fs2/errors.Errorf(0x6e3f00, 0x4b, 0xc829ce47b0, 0x4, 0x4, 0x0, 0x0)
        /home/hendersont/stuff/research/sfp/src/github.com/timtadh/fs2/errors/error.go:16 +0x89
github.com/timtadh/fs2/bptree.(*internal).updateK(0x7f66a01dd000, 0x0, 0x22, 0x7f66b43b7020, 0x4, 0xfe0, 0x0, 0x0)
        /home/hendersont/stuff/research/sfp/src/github.com/timtadh/fs2/bptree/internal.go:229 +0x5ff
github.com/timtadh/fs2/bptree.(*BpTree).internalDelete.func4.1(0x7f66b43b7020, 0x4, 0xfe0, 0x0, 0x0)
        /home/hendersont/stuff/research/sfp/src/github.com/timtadh/fs2/bptree/remove.go:118 +0x63
github.com/timtadh/fs2/bptree.(*BpTree).firstKey.func2(0x7f66b43b7000, 0x0, 0x0)
        /home/hendersont/stuff/research/sfp/src/github.com/timtadh/fs2/bptree/first.go:20 +0xed
github.com/timtadh/fs2/bptree.(*BpTree).do.func1(0x7f66b43b7000, 0x1000, 0x1000, 0x0, 0x0)
        /home/hendersont/stuff/research/sfp/src/github.com/timtadh/fs2/bptree/do.go:103 +0xe7
github.com/timtadh/fs2/fmap.(*BlockFile).Do(0xc8251ab440, 0x142bb000, 0x1, 0xc829ce49c0, 0x0, 0x0)
        /home/hendersont/stuff/research/sfp/src/github.com/timtadh/fs2/fmap/fmap.go:563 +0xb6
github.com/timtadh/fs2/bptree.(*BpTree).do(0xc82636ff80, 0x142bb000, 0xc829ce4a20, 0xc829ce4a10, 0x0, 0x0)
        /home/hendersont/stuff/research/sfp/src/github.com/timtadh/fs2/bptree/do.go:107 +0x69
github.com/timtadh/fs2/bptree.(*BpTree).firstKey(0xc82636ff80, 0x142bb000, 0xc829ce4a60, 0x0, 0x0)
        /home/hendersont/stuff/research/sfp/src/github.com/timtadh/fs2/bptree/first.go:22 +0x6f
github.com/timtadh/fs2/bptree.(*BpTree).internalDelete.func4(0x7f66a01dd000, 0x0, 0x0)
        /home/hendersont/stuff/research/sfp/src/github.com/timtadh/fs2/bptree/remove.go:119 +0x93
github.com/timtadh/fs2/bptree.(*BpTree).do.func1(0x7f66a01dd000, 0x1000, 0x1000, 0x0, 0x0)
        /home/hendersont/stuff/research/sfp/src/github.com/timtadh/fs2/bptree/do.go:101 +0x98
github.com/timtadh/fs2/fmap.(*BlockFile).Do(0xc8251ab440, 0xe1000, 0x1, 0xc829ce4bb0, 0x0, 0x0)
        /home/hendersont/stuff/research/sfp/src/github.com/timtadh/fs2/fmap/fmap.go:563 +0xb6
github.com/timtadh/fs2/bptree.(*BpTree).do(0xc82636ff80, 0xe1000, 0xc829ce4d40, 0x6f9da0, 0x0, 0x0)
        /home/hendersont/stuff/research/sfp/src/github.com/timtadh/fs2/bptree/do.go:107 +0x69
github.com/timtadh/fs2/bptree.(*BpTree).doInternal(0xc82636ff80, 0xe1000, 0xc829ce4d40, 0x0, 0x0)
        /home/hendersont/stuff/research/sfp/src/github.com/timtadh/fs2/bptree/do.go:43 +0x4d
github.com/timtadh/fs2/bptree.(*BpTree).internalDelete(0xc82636ff80, 0x0, 0xe1000, 0xc81c000, 0xc823239630, 0x4, 0x4, 0xc829ce4f70, 0x142bb000, 0x0, ...)
        /home/hendersont/stuff/research/sfp/src/github.com/timtadh/fs2/bptree/remove.go:120 +0x894
github.com/timtadh/fs2/bptree.(*BpTree).delete(0xc82636ff80, 0x0, 0xe1000, 0x0, 0xc823239630, 0x4, 0x4, 0xc829ce4f70, 0xc823239630, 0x0, ...)
        /home/hendersont/stuff/research/sfp/src/github.com/timtadh/fs2/bptree/remove.go:58 +0x120
github.com/timtadh/fs2/bptree.(*BpTree).remove(0xc82636ff80, 0xe1000, 0xc823239630, 0x4, 0x4, 0xc829ce4f70, 0x4, 0x4, 0x0, 0x0)
        /home/hendersont/stuff/research/sfp/src/github.com/timtadh/fs2/bptree/remove.go:35 +0x83
github.com/timtadh/fs2/bptree.(*BpTree).Remove(0xc82636ff80, 0xc823239630, 0x4, 0x4, 0xc829ce4f70, 0x0, 0x0)
        /home/hendersont/stuff/research/sfp/src/github.com/timtadh/fs2/bptree/remove.go:25 +0x6d
github.com/timtadh/fs2/bptree.(*Varchar).szDel(0xc8224c2780, 0xa4, 0x14c4ef5c, 0x0, 0x0)
        /home/hendersont/stuff/research/sfp/src/github.com/timtadh/fs2/bptree/varchar.go:413 +0x2e3
github.com/timtadh/fs2/bptree.(*Varchar).indexRemove(0xc8224c2780, 0xa4, 0x14c4ef5c, 0x0, 0x0)
        /home/hendersont/stuff/research/sfp/src/github.com/timtadh/fs2/bptree/varchar.go:425 +0x5c
github.com/timtadh/fs2/bptree.(*Varchar).Alloc(0xc8224c2780, 0x8c, 0x14c4ef5c, 0x0, 0x0)
        /home/hendersont/stuff/research/sfp/src/github.com/timtadh/fs2/bptree/varchar.go:220 +0x52e
github.com/timtadh/fs2/bptree.(*BpTree).checkValue(0xc82636ffc0, 0xc8294ad950, 0x8c, 0x8c, 0x0, 0x0, 0x0, 0x0, 0x0)
        /home/hendersont/stuff/research/sfp/src/github.com/timtadh/fs2/bptree/insert.go:16 +0x9e
github.com/timtadh/fs2/bptree.(*BpTree).Add(0xc82636ffc0, 0xc8294a4cf0, 0x88, 0x88, 0xc8294ad950, 0x8c, 0x8c, 0x0, 0x0)
        /home/hendersont/stuff/research/sfp/src/github.com/timtadh/fs2/bptree/insert.go:51 +0x254
github.com/timtadh/sfp/stores/bytes_subgraph.(*BpTree).Add(0xc8251ab4c0, 0xc8294a4cf0, 0x88, 0x88, 0xc82ee45200, 0x0, 0x0)
        /home/hendersont/stuff/research/sfp/src/github.com/timtadh/sfp/stores/bytes_subgraph/wrapper.go:237 +0x16a

Great work! Question regarding compaction.

Possible to add a feature to compact the file when key/values are deleted?

or is it built in already?

Is this production ready?

1. Is this production ready?
2. Any other limitations like size? what's the max it can go?

Can memory be smaller than disk size?

1. Can memory be smaller than disk size? what happens when it does?
2. do you have an option where the fsync is immediate?

1mb for creating an empty db is too large

1. can u make it nothing since i didnt even update any information inside.

2. bolt uses 16kb for nothing. i actually expect this to be like 100bytes and below

if i want to access a key from a file, does it read the whole file into mem or just that particular key section? just wondering if i need to have lots of mem available for this. 100GB per file size advisable etc?

if i want to access a key from a file, does it read the whole file into mem or just that particular key section? just wondering if i need to have lots of mem available for this. 100GB per file size advisable etc?

does it reclaim disk space with deleting keys since this is btree

How to flush bpt? reset / clear everything?

How to flush bpt? reset / clear everything?

Great work! i think there's an area that can improve. not sure if u will accept pull request?

bpt.Remove is a bit slow. Any ways to speed it up?

    bpt.Remove(buf, func(value []byte) bool {
            return true
    })

any other function / possibility to remove quicker?