dc-fukuoka / pfind Goto Github PK

pfind: A simple parallel find command which is parallelized by OpenMP.
How to use:
$ pfind <search directory> <filename>

Cuerrent limitations:
  - Asterisk can be used for <filename>, but not for <search directory>.
  - Only file can be searched.

Performance compareson (on Intel(R) Xeon(R) CPU E5-2680 0 @ 2.70GHz):
--------------------------
== GNU find(serial) ==
[fukuoka@rokuchan85 ~]:25024>time find /work/workdir1/fukuoka/HPCRT/ -type f -name "HPCRT2985.tgz"
/work/workdir1/fukuoka/HPCRT/2985/tmp/HPCRT2985.tgz

real    0m17.884s
user    0m1.630s
sys     0m6.423s

== GNU find + xargs ==
[fukuoka@rokuchan85 ~]:25007>time gnupfind /work/workdir1/fukuoka/HPCRT/ "HPCRT2985.tgz"
/work/workdir1/fukuoka/HPCRT/2985/tmp/HPCRT2985.tgz

real    0m22.428s
user    0m3.505s
sys     0m15.301s

== pfind ==
[fukuoka@rokuchan85 ~]:25008>time pfind /work/workdir1/fukuoka/HPCRT/ "HPCRT2985.tgz"
/work/workdir1/fukuoka/HPCRT/2985/tmp/HPCRT2985.tgz

real    0m7.382s
user    0m2.421s
sys     0m6.941s

== just a reference, linked list version of pfind ==
[fukuoka@rokuchan85 fukuoka]:25129>time pfind.list /work/workdir1/fukuoka/HPCRT/ "HPCRT2985.tgz"
/work/workdir1/fukuoka/HPCRT/2985/tmp/HPCRT2985.tgz

real    0m11.745s
user    0m35.596s
sys     0m4.239s

-- gnupfind(): a simple find bash function by GNU find + xargs
[fukuoka@rokuchan85 ~]:25021>type gnupfind
gnupfind is a function
gnupfind ()
{
    dir=$1;
    filename=$2;
    ncores=$((`cut -d"-" -f2 /sys/devices/system/cpu//online`+1));
    found=`find $dir -maxdepth 1 -type f -name $filename`;
    if [ -n $found ] && [ "x$found" != "x" ]; then
        echo $found;
    else
        find $dir -maxdepth 1 -type d | xargs -P$ncores -IXXX find XXX -type f -name $filename | sort -u;
    fi
}
--------------------------