Implement Parallel readdirp in dht

[gluster]

Implement Parallel readdirp in dht

[ShyamsundarR]

Marking this for 3.11 as this is not making into the 3.10 release

[poornimag]

The actual feature patch is already part of 3.10 tag[1]. The 3 patches remaining are bug fixes [2] which can be back-ported?. Hence for 3.10 this feature can still go as experimental ?

[1] https://review.gluster.org/#/c/16072/
[2] https://review.gluster.org/#/q/status:open+project:glusterfs+branch:master+topic:bug-1401812

[ShyamsundarR]

I was tracking more than just that review for this issue, hence moved it out. Correcting the same now.

[poornimag]

Please review the following release Notes:
Directory enumeration performance improvement (Experimental):
Currently the directory listing gets slower as the number of bricks/nodes increases in a volume, though the file/directory numbers remain unchanged. With this feature, the performance of directory listing is made mostly independent of the number of nodes/bricks in the volume. Thus scale doesn't exponentially reduce the directory listing performance. (On a 2, 5, 10, 25 brick setup we saw ~5, 100, 400, 450% improvement consecutively)
To enable this feature:
# gluster volume set <VOLNAME> performance.readdir-ahead on
# gluster volume set <VOLNAME> performance.parallel-readdir on
To disable this feature:
# gluster volume set <VOLNAME> performance.parallel-readdir off

If there are more than 50 bricks in the volume it is good to increase the cache size to be more than 10Mb (default value):
# gluster volume set <VOLNAME> performance.rda-cache-limit <CACHE SIZE>

[ShyamsundarR]

Feature is part of release-3.10

Release notes updated: https://review.gluster.org/16543

[ShyamsundarR]

@nixpanic why is this moved out of release-3.10 project lane? This was delivered in 3.10? I am marking it back as 3.10, if you are moving it again, drop a note here for clarity. Thanks.