ZipStore performance

[zarr-developers]

ZipStore performance

[alimanfoo]

Here's some profiling data on an operation that takes 15 milliseconds with a DirectoryStore:

         24842632 function calls in 16.836 seconds

   Ordered by: cumulative time

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.000    0.000   16.836   16.836 {built-in method builtins.exec}
        1    0.000    0.000   16.836   16.836 <string>:1(<module>)
        1    0.001    0.001   16.836   16.836 core.py:292(__getitem__)
       20    0.302    0.015   16.834    0.842 core.py:539(_chunk_getitem)
       20    0.000    0.000   16.518    0.826 storage.py:732(__getitem__)
       20    0.000    0.000   16.515    0.826 zipfile.py:981(__init__)
       20    6.168    0.308   16.514    0.826 zipfile.py:1085(_RealGetContents)
  1307400    3.530    0.000    4.484    0.000 {method 'decode' of 'bytes' objects}
  1307380    1.774    0.000    2.376    0.000 zipfile.py:436(_decodeExtra)
  1307380    1.780    0.000    2.302    0.000 zipfile.py:320(__init__)
  1307400    0.427    0.000    0.954    0.000 cp437.py:14(decode)
  3922200    0.806    0.000    0.806    0.000 {built-in method _struct.unpack}
  5229520    0.599    0.000    0.599    0.000 {method 'read' of '_io.BytesIO' objects}
  1307400    0.527    0.000    0.527    0.000 {built-in method _codecs.charmap_decode}
  1307380    0.317    0.000    0.317    0.000 {method 'find' of 'str' objects}
  5229623    0.270    0.000    0.270    0.000 {built-in method builtins.len}
  1307380    0.205    0.000    0.205    0.000 {built-in method builtins.chr}
  1307420    0.087    0.000    0.087    0.000 {method 'append' of 'list' objects}