URE performance

[opencog]

URE performance

[linas]

An obvious "quick fix" would be to alter the pattern matcher callbacks to simply not consider any atoms that are not in a focus-set. This is quite easy to do. It's somewhat similar to the attention-based variant of the pattern matcher, which avoids looking at atoms with low AV.

A focus-set variant of the matcher could provide the determinism that the current chainers have, and surely run faster than the current chainers. It would also avoid some of the indeterminism associated with the AV subsystem.

[ngeiswei]

The forward chainer is rather messy as compared to the new backward chainer. I intend as soon as I start running the large experiments on the BC to re-organize the FC code to be cleaner and ultimately faster.

[linas]

I wasn't worried about the mess, per-se, more about the copy-in and copy-out of the focus-set. This affects the backwards-chainer too, not just the forwards-chainer. (right? because BC also does this copy-in?)

I deleted my other comments and moved them to issue #26..

[ngeiswei]

The BC does no copying. There is little in common between the BC and FC. The BC has been re-written from scratch and, although not being optimized at a low level, is more thoughtfully designed. I'll soon revisit and simplify the FC, so I'd advice not to spend too much on it yet.

[ivan-ushakov]

Am I right that problem still here? I mean DefaultPatternMatchCB::eval_term() still creates new atomspace each time and profiler shows that it has huge impact on performance.

[ngeiswei]

Thanks @ivan-ushakov for your interest, however as I said the FC is soon gonna be redesigned. It's gonna be merged to the BC to form a mixed chainer, and its main routines exposed so that the user can easily build his/her own chainer. So unless you or other users have a strong immediate incentive to have an optimized FC I wouldn't bother.

[ngeiswei]

@ivan-ushakov that is said, any profiling results you have would still be of interest, even if it's about FC, so feel free to paste them here. If you happen to do some profiling on the BC that would be of even more interest. Thanks!

[linas]

There is supposed to be a pool of empty atom-spaces, ready to be used, and so grabbing one should be very fast ... is this not the case? Maybe there is an overhead for the first one or two being created, but after that, they are supposed to be recycled ... Curtis implemented this, exactly to avoid the heavy overhead of creating them over and over.

[linas]

(and, no, we probably do not have a unit test to make sure that this pool is working as designed. Writing such a unit test should be easy -- first, measure the time to create 100 normal atomspaces, then measure the time to create 100 temp atomspaces -- the second time should be at least 10x faster or maybe 1000x faster than the first)

[ngeiswei]

I misread @ivan-ushakov message, indeed it's not about FC profiling per se. @ivan-ushakov, could you be more specific? How and what did you profile?

[ivan-ushakov]

I'm talking about method DefaultPatternMatchCB::eval_term() and this comment inside it:

Evaluation of the link requires working with an atomspace of some sort, so that the atoms can be communicated to scheme or python for the actual evaluation. We don't want to put the proposed grounding into the "real" atomspace, because the grounding might be insane. So we put it here. This is probably not very efficient, but will do for now...

Also about caching. I found that key CACHED_IMPLICATOR not enabled by default. So am I right that we don't have this cache if we do usual build?