Callback interface is expensive

[venemo]

Callback interface is expensive

[Venemo]

@Matt-Esch Which methods are you talking about, specifically?

[Venemo]

@russtee I'm looking forward to getting your feedback about this as well. :)

[russtee]

We use lmdb synchronously for all operations whenever possible and the extra code overhead to handle the cursor callbacks seems unnecessary.

LMDB is very fast especially when running on SSD's.

That said on slower machines callbacks could be handy and would fit well into the node streams system for iteration operations.

[Venemo]

One way to go about this is to create a JavaScript object and return that instead of calling a callback. The proposed object could have a key and data property. Would something like this suit your needs?

For example,

var x = cursor.getCurrentString();
console.log(x.key, x.data);

[russtee]

that would work. However do I actually need the key as I would most likely already have it from the cursor iteration function?

[Venemo]

What do you mean? Can you post some example code of how you would be comfortable using it?

[russtee]

for (var c = cursor.goToFirst(); c; c=cursor.goToNext()) {
   var val = cursor.getCurrentBinary();

As a hack/test I changed my local copy of the library to return the value instead of the key when 'convertFunc' was set.

Probably not they way you would want to do it but it seems to work and solves the problem we have.

[Venemo]

But then this would make it impossible to see the current key.

[russtee]

We use the value from goToNext etc. as we always call that prior.

[Venemo]

I see. Okay, let me think about it a bit more and I'll get back to you guys.

[rneilson]

Let me +1 just returning the value from getCurrentWhichever() -- you'll always have the current key as the return value of cursor.goToWhichever(), and the callback makes any iteration (by loop, or especially by forEach(), map(), reduce(), etc) much clumsier.

[rneilson]

I've submitted PR #78, which I think is a workable compromise. The callback interface is still present, if you need both key and value together, but getCurrentString() and friends return the value instead of the key regardless, meaning they can be used without a callback if desired. Also, there's no extra conversion overhead, since I merely relocated the call to convertFunc(cw->data).

[Venemo]

Thank you @rneilson for your work. This is much better than what we had before, so it's a step in the right direction. :)

[rneilson]

@Venemo You're quite welcome.

[Venemo]

I believe we can agree to close this issue now, the current API doesn't require the use of callback anymore.