cgohlke/czifile

Read Carl Zeiss(r) Image (CZI) files

Read Carl Zeiss(r) Image (CZI) files

Czifile is a Python library to read Carl Zeiss Image (CZI) files, the native file format of the ZEN(r) software by Carl Zeiss Microscopy GmbH. CZI files contain multidimensional images and metadata from microscopy experiments.

Author:	Christoph Gohlke
Organization:	Laboratory for Fluorescence Dynamics. University of California, Irvine
License:	3-clause BSD
Version:	2019.7.2

Requirements

• CPython 2.7 or 3.5+
• Numpy 1.14
• Tifffile 2019.7.2
• Imagecodecs 2019.5.22 (optional; used for decoding LZW, JPEG, and JPEG XR)

Revisions

2019.7.2

Require tifffile 2019.7.2.

2019.6.18

Add package main function to view CZI files. Fix BGR to RGB conversion. Fix czi2tif conversion on Python 2.

2019.5.22

Fix czi2tif conversion when CZI metadata contain non-ASCII characters. Use imagecodecs_lite as a fallback for imagecodecs. Make CziFile.metadata a function (breaking). Make scipy an optional dependency; fallback on ndimage or fail on zoom().

2019.1.26

Fix czi2tif console script. Update copyright year.

2018.10.18

Rename zisraw package to czifile.

2018.8.29

Move czifile.py and related modules into zisraw package. Move usage examples to main docstring. Require imagecodecs package for decoding JpegXrFile, JpgFile, and LZW.

2018.6.18

Save CZI metadata to TIFF description in czi2tif. Fix AttributeError using max_workers=1. Make Segment.SID and DimensionEntryDV1.dimension str types. Return metadata as XML unicode string or dict, not etree. Return timestamps, focus positions, events, and luts as tuple or ndarray

2017.7.21

Use multi-threading in CziFile.asarray to decode and copy segment data. Always convert BGR to RGB. Remove bgr2rgb options. Decode JpegXR directly from byte arrays.

2017.7.13

Add function to convert CZI file to memory-mappable TIFF file.

2017.7.11

Add 'out' parameter to CziFile.asarray. Remove memmap option from CziFile.asarray (breaking). Change spline interpolation order to 0 (breaking). Make axes return a string. Require tifffile 2017.7.11.

2014.10.10

Read data into a memory mapped array (optional).

2013.12.4

Decode JpegXrFile and JpgFile via _czifle extension module. Attempt to reconstruct tiled mosaic images.

2013.11.20

Initial release.

Notes

The API is not stable yet and might change between revisions.

Python 2.7 and 3.4 are deprecated.

"ZEISS" and "Carl Zeiss" are registered trademarks of Carl Zeiss AG.

The ZISRAW file format design specification [1] is confidential and the license agreement does not permit to write data into CZI files.

Only a subset of the 2016 specification is implemented. Specifically, multifile images, image pyramids, and topography images are not yet supported.

Tested on Windows with a few example files only.

Czifile relies on the imagecodecs package for decoding LZW, JPEG, and JPEG XR compressed images. Alternatively, the imagecodecs_lite package can be used for decoding LZW compressed images.

Other libraries for reading CZI files (all GPL licensed):

• libCZI
• Python-bioformats
• Pylibczi

References

1. ZISRAW (CZI) File Format Design Specification Release Version 1.2.2. CZI 07-2016/CZI-DOC ZEN 2.3/DS_ZISRAW-FileFormat.pdf (confidential). Documentation can be requested at https://www.zeiss.com/microscopy/us/products/microscope-software/zen/czi.html

Examples

Read image data from a CZI file as numpy array:

>>> image = imread('test.czi')
>>> image.shape
(3, 3, 3, 250, 200, 3)
>>> image[0, 0, 0, 0, 0]
array([10, 10, 10], dtype=uint8)

czifile's Issues

Convert .czi to .tif

thanks for your great work. In my project, i want to process wsi image with an python-lib openslide, which support the .svs or .tif format. how can i convert .czi to .tif file by this lib? (it will be good if there are some code snippet)

Suggestion: add a get_metadata function

Very useful package!
However, it would be extremely nice to be able to retrieve the identity of each image dimension. In this way, a get_metadata function could be used with the imread function so each dimension of the multidimensional image could be identified.
Thanks!

imread error when passing a string

Hi @cgohlke ,

I'm retrieving an error when passing a string to the imread function and presume it's a bug. This example works well:

import czifile
from pathlib import Path
image = czifile.imread(Path("../../data/PupalWing.czi"))

But this throws an error:

import czifile
image = czifile.imread("../../data/PupalWing.czi")

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Cell In[12], line 2
      1 import czifile
----> 2 image = czifile.imread("../../data/PupalWing.czi")

File ~\mambaforge\envs\bio39\lib\site-packages\czifile\czifile.py:220, in imread(filename, *args, **kwargs)
    214 def imread(filename, *args, **kwargs):
    215     """Return image data from CZI file as numpy array.
    216 
    217     'args' and 'kwargs' are arguments to the CziFile.asarray function.
    218 
    219     """
--> 220     with CziFile(filename) as czi:
    221         result = czi.asarray(*args, **kwargs)
    222     return result

File ~\mambaforge\envs\bio39\lib\site-packages\czifile\czifile.py:275, in CziFile.__init__(self, arg, multifile, filesize, detectmosaic)
    272 if multifile and self.header.file_part and isinstance(arg, basestring):
    273     # open master file instead
    274     self._fh.close()
--> 275     name, _ = match_filename(arg)
    276     self._fh = FileHandle(name)
    277     self.header = Segment(self._fh, 0).data()

File ~\mambaforge\envs\bio39\lib\site-packages\czifile\czifile.py:1148, in match_filename(filename)
   1145 match = re.search(r'(.*?)(?:\((\d+)\))?\.czi$',
   1146                   filename, re.IGNORECASE).groups()
   1147 name = match[0] + '.czi'
-> 1148 part = int(match[1]) if len(match) > 1 else 0
   1149 return name, part

TypeError: int() argument must be a string, a bytes-like object or a number, not 'NoneType'

If you want to reproduce this, you can download the example image here.

Thanks!

Best,
Robert

Load czi file is not deterministic

Thank you release the tool. We found that sometimes using czifile package to load .czi file is not deterministic.

czi = czifile.CziFile(file_path)
meta = czi.metadata(raw=False)
channel_info = meta['ImageDocument']['Metadata']['DisplaySetting']['Channels']['Channel']
if type(channel_info) == list:
    channel_names = [c['Name'] for c in channel_info]
    channel_index = channel_names.index('EGFP')
else:
    channel_index = 0
dim_indices = [':'] * len(czi.axes)
dim_indices[czi.axes.index('C')] = 'channel_index'
image = eval('czi.asarray(resize=False)[%s].squeeze()' % ','.join(dim_indices))

We run the same code (above) twice and get different arrays. It seems a bug in this package. We recommend using aicspylibczi to avoid the problem.

Handling escaped XML metadata

A CZI file has this metadata for a subblock:

<METADATA><Tags><AcquisitionTime>2021-03-03T12:51:11.0123443Z</AcquisitionTime><ImageScaling>&lt;ImageScaling&gt;
  &lt;ImagePixelSize&gt;6.5,6.5&lt;/ImagePixelSize&gt;
&lt;/ImageScaling&gt;</ImageScaling><DetectorState>&lt;CameraState&gt;
  &lt;ApplyCameraProfile&gt;false&lt;/ApplyCameraProfile&gt;
  &lt;ApplyImageOrientation&gt;true&lt;/ApplyImageOrientation&gt;
  &lt;ExposureTime&gt;80005705.882353&lt;/ExposureTime&gt;
  &lt;Frame&gt;128,128,2048,2048&lt;/Frame&gt;
  &lt;ImageOrientation&gt;3&lt;/ImageOrientation&gt;
&lt;/CameraState&gt;</DetectorState><StageXPosition>+000000209505.8000</StageXPosition><StageYPosition>+000000045936.2000</StageYPosition><FocusPosition>+000000021667.7820</FocusPosition><RoiCenterOffsetX>+000000000000.0000</RoiCenterOffsetX><RoiCenterOffsetY>+000000000000.0000</RoiCenterOffsetY></Tags><DataSchema><ValidBitsPerPixel>16</ValidBitsPerPixel></DataSchema><AttachmentSchema /></METADATA>

Because part of the XML contents is escaped, it remains text when the metadata is converted to JSON, just as expected. Do you have suggestions for dealing with such files?

Lack of zstd compression support

It looks like CZI file format was updated to support compressing data using zstd that is no supported by this package yet.

based on my investigation they added two modes. 5: zstd0 (plain zstd, but do not have such file to test)
https://github.com/ZEISS/libczi/blob/4a60e22200cbf0c8ff2a59f69a81ef1b2b89bf4f/Src/libCZI/decoder_zstd.cpp#L39

And mode 6 named zstd1

https://github.com/ZEISS/libczi/blob/4a60e22200cbf0c8ff2a59f69a81ef1b2b89bf4f/Src/libCZI/decoder_zstd.cpp#L39

So it seems like to support mode 5 it will be enough to add

DECOMPRESS[5] = imagecodecs.zstd_decode

to this dict

czifile/czifile/czifile.py

Lines 1227 to 1231 in a70265f

	if imagecodecs is not None:
	DECOMPRESS[2] = imagecodecs.lzw_decode
	if hasattr(imagecodecs, 'jpeg_decode'):
	DECOMPRESS[1] = imagecodecs.jpeg_decode
	DECOMPRESS[4] = imagecodecs.jxr_decode

Mode 6 requires to add parsing header and decoding that may be done with such a code:

class ZSTD1Header(typing.NamedTuple):
    """
    ZSTD1 header structure
    based on:
    https://github.com/ZEISS/libczi/blob/4a60e22200cbf0c8ff2a59f69a81ef1b2b89bf4f/Src/libCZI/decoder_zstd.cpp#L19
    """

    header_size: int
    hiLoByteUnpackPreprocessing:bool


def parse_zstd1_header(data, size):
    """
    Parse ZSTD header

    https://github.com/ZEISS/libczi/blob/4a60e22200cbf0c8ff2a59f69a81ef1b2b89bf4f/Src/libCZI/decoder_zstd.cpp#L84
    """
    if size < 1:
        return ZSTD1Header(0, False)

    if data[0] == 1:
        return ZSTD1Header(1, False)

    if data[0] == 3 and size < 3:
        return ZSTD1Header(0, False)

    if data[1] == 1:
        return ZSTD1Header(3, bool(data[2] & 1))

    return ZSTD1Header(0, False)


def decode_zstd1(data):
    """
    Decode ZSTD1 data
    """
    header = parse_zstd1_header(data, len(data))
    return imagecodecs.zstd_decode(data[header.header_size:])

(this code does not support hilo byte unpacking, I do not dig it yet).

Hovewer provide own decode function is not enough, as zstd_decode return bytes, not array.

So it require update also this line:

czifile/czifile/czifile.py

Line 660 in a70265f

if de.compression == 2:

to

if de.compression in {2, 5, 6}:

The current example file that I had is 500mb so require an alternative way to share. I try to obtain a smaller one.

Did you prefer to do this on your site or PR?

images with scenes greater than 0 cannot be processed

I am using czifile (2019.7.2) with python 3.6.12. I can easily read in stacked image files or single image files using czifile.imread as long as the scene in the header file is 0.

Batch scans are now being collected on the Zeiss which include scanning multiple images as one file and then using the Zen software to split the scan into individual images. This process attaches a scene integer to each split image. The software collecting the images has not changed, just the imaging process.

Is there any way to get around this?

For reference:
OS: Win10 v 1909
numpy==1.14
tifffile==2019.7.2
imagecodecs==2019.5.22

Perf report while reading CZI file

Hello Christoph,

I have a question. Python is printing constantly this report while reading a CZI file:

`* Perf Report

---

Image Width = 1600, Height = 1200, total MegaPixels = 1.9 MP
m_ptEncDecPerf (excl I/O): 2475.000 milliseconds, 0.775758 MP/sec
*** WARNING: 360 time intervals were measured as zero. This perf timer has insufficient precision!
m_ptEndToEndPerf (incl I/O): 3303.000 milliseconds, 0.581290 MP/sec`

Do you have any idea what's all about this and how to supress this report?

Windows 10
Python 3.6
czifile 2019.7.2
Thanks again for your efforts with czifile,
S.

problem with czi2tif

When I use the czi2tif function, I find that the output of any image(wsi or czi file ) is black. I am confused about this and don't know what the reason is. thanks!

Deprecation warning for scipy zoom import

Hi,
When importing czifile I get a deprecation warning for this line:

from scipy.ndimage.interpolation import zoom

DeprecationWarning: Please import `zoom` from the `scipy.ndimage` namespace; the `scipy.ndimage.interpolation` namespace is deprecated and will be removed in SciPy 2.0.0.