Stream ordering of buckets.streamClosestPeers(targetId) does not follow the xor ordering (Spec Mismatch)

[consensys]

Stream ordering of buckets.streamClosestPeers(targetId) does not follow the xor ordering (Spec Mismatch)

[ajsutton]

The optimised version of this makes sense to me except that we then can't stream the results effectively. There may be use cases where we do know the number of results required, but for Teku we need to find some number of nodes meeting a specific criteria which may require scanning more nodes. That would then essentially devolve to the unoptimised case where we have to get every node to ensure we actually get enough once filtered.

It would be doable though if we passed in a Predicate to allow filtering the nodes (potentially out of order).

[SangameswaranRS]

Yes, Totally agree with you @ajsutton Actually I faced the same myself.

It totally makes sense as we need some ACTIVE k peers, we won't be knowing how much we should traverse in order to get k

Also, its a great insight to pass a Predicate, so that we'd know if we have streamed enough from the buckets.

Also I faced a weird case when streaming buckets the way I posited. For small values of N its working fine, (Tested over many iterations), but for larger values of N there is sometimes a node which is closer in bucket b+1 than b. (Since they share exactly cpl prefix bits), but there are some, cases like CPL bits....mismatch..mismatch vs CPL bits .... mismatch ..match , leading the xor distance to be smaller than bucket b+1.

Yeah, anyway we need to sort, the go-libp2p-kbucket guys also sort at the end to handle cases like this. So I think It makes sense only if we limit our traversing when we have enough peers.