ome/ome-files-cpp

OME Files is a C++ library for reading and writing data using the open and standard OME-TIFF file format using the OME data model

OME Files C++

OME Files is a standalone C++ library for reading and writing life sciences image file formats. This project has been archived while seeking further funding.

**Independent development is taking place under https://gitlab.com/codelibre/ome-files **

Links

• Overview of all components
• Documentation
• Tutorial
• API reference
• Performance testing

Purpose

OME Files' primary purpose is to convert proprietary microscopy data into an open standard called the OME data model, particularly into the OME-TIFF file format. See the statement of purpose for a thorough explanation and rationale. OME Files provides support for reading and writing files using the OME-TIFF file format and for the OME metadata model which is the basis for the file format.

Supported formats

OME-TIFF using the OME metadata model.

For users

Many software packages use Bio-Formats to read and write open microscopy formats such as OME-TIFF in Java programs. OME Files provides equivalent functionality for C++ programs.

For developers

You can use OME Files C++ to easily support OME-TIFF in your software.

More information

For more information, see the documentation.

Pull request testing

We welcome pull requests from anyone, but ask that you please verify the following before submitting a pull request:

• verify that the branch merges cleanly into master
• verify that the branch compiles
• run the unit tests (ctest -V) and correct any failures
• make sure that your commits contain the correct authorship information and, if necessary, a signed-off-by line
• make sure that the commit messages or pull request comment contains sufficient information for the reviewer(s) to understand what problem was fixed and how to test it

ome-files-cpp's Issues

Unhandled model references

With a current build, I'm seeing lots of this type of warning/error:

[warning] Unable to handle reference of type: St10shared_ptrIN3ome3xml5model9ReferenceEE

That's std::shared_ptr<ome::xml::model::Reference> in English.

That's indicative of something wrong in the model code, or invalid input. The former is most likely. Either way, it's something which should be investigated and fixed. Has there been a change to the model without a corresponding update to the C++ model code or templates?

Investigate writing tiles out of order

The current behaviour when writing a plane is to flush tiles sequentially. This was, I think, done for the purpose of strict compatibility with the Bio-Formats TiffWriter/TiffSaver which strictly require writing in order. Looking at the libtiff documentation, libtiff does not have this restriction, and it should be possible to flush files as soon as they are completed. It would even be possible to support rewriting. This would require the IFD flush method to be adjusted to update the behaviour for coverage checking and tile ordering.

See this mail for further details.

ome-files does not correctly load dependencies when imported in CMake

I'm trying to integrating ome-files in a CMake C++ project. In my hands, the following is a minimal CMakeLists.txt to successfully build a ome-files project. I am building on macOS 10.12.6 with clang 4.0.

cmake_minimum_required(VERSION 3.9)
project(ome-test) 

set(CMAKE_CXX_STANDARD 11)

set(CMAKE_PREFIX_PATH "$ENV{OME_FILES_ROOT}")

find_package(Boost COMPONENTS log iostreams) # <- Build fails without this line
find_package(OMEXML REQUIRED)
find_package(OMEFiles REQUIRED)

include_directories($ENV{OME_FILES_ROOT}/include)
add_executable(ome-test main.cpp)
target_link_libraries(ome-test OME::Files)

If I exclude the find_package(Boost COMPONENTS log iostreams) line (or use different boost components) CMake fails with the following error:

CMake Error at CMakeLists.txt:14 (add_executable):
  Target "ome-test" links to target "Threads::Threads" but the target was not
  found.  Perhaps a find_package() call is missing for an IMPORTED target, or
  an ALIAS target is missing?


CMake Error at CMakeLists.txt:14 (add_executable):
  Target "ome-test" links to target "Threads::Threads" but the target was not
  found.  Perhaps a find_package() call is missing for an IMPORTED target, or
  an ALIAS target is missing?


CMake Error at CMakeLists.txt:14 (add_executable):
  Target "ome-test" links to target "Boost::log_setup" but the target was not
  found.  Perhaps a find_package() call is missing for an IMPORTED target, or
  an ALIAS target is missing?


CMake Error at CMakeLists.txt:14 (add_executable):
  Target "ome-test" links to target "Boost::log" but the target was not
  found.  Perhaps a find_package() call is missing for an IMPORTED target, or
  an ALIAS target is missing?


CMake Error at CMakeLists.txt:14 (add_executable):
  Target "ome-test" links to target "Boost::iostreams" but the target was not
  found.  Perhaps a find_package() call is missing for an IMPORTED target, or
  an ALIAS target is missing?

[SNIP]

CMake Error at CMakeLists.txt:14 (add_executable):
  Target "ome-test" links to target "Boost::filesystem" but the target was
  not found.  Perhaps a find_package() call is missing for an IMPORTED
  target, or an ALIAS target is missing?

As far as I understand, find_package(OMEFiles REQUIRED) should automatically import its dependencies and this line should not be required. This error occurs when using either the downloaded ome-files build or when using a freshly compiling super-build from head.

I have the build working now but thought I'd let you know as it look a while to figure this out.

Version.h gets stale on branch switch

Consider the following steps:

• git clone [email protected]:ome/ome-files-cpp.git
• mkdir build
• cd build
• cmake ..
• make
• grep OME_FILES_VCS_SHORTREVISION lib/ome/files/Version.h
  #define OME_FILES_VCS_SHORTREVISION "3a25964"

Now if you change branches in the repo which has a different HEAD commit and rebuild with make and call grep OME_FILES_VCS_SHORTREVISION lib/ome/files/Version.h again the output has not changed.

A fix could be to regenerate Version.h at every make invocation.

make doc-pdf fails

I've compiled ome-files-cpp on my debian jessie machine.
The creation of the pdf manual was not successfull, this stopped the build process.
See make-doc.txt for the build log.

$doxygen --version
1.8.13
$sphinx-build --version
Sphinx (sphinx-build) 1.5a1

Clarify usage of sizeC in the header

see:

• ome-files-cpp/lib/ome/files/CoreMetadata.h

  Line 89 in e5a4d5c

  /// Number of channels.

• http://lists.openmicroscopy.org.uk/pipermail/ome-devel/2018-April/004183.html

Reading metadata for small files can consume lots of memory

Using the simple zzuf fuzzer I tried feeding fuzzed images into the info tool shipped with ome-files-cpp and found that large ImageWidth and ImageLength parameters in the TIFF header will cause info to attempt to allocate very large amounts of memory, leading to OOM kills in my testing VM, even for very small input files. Anecdotally, the libtiff tiffinfo tool also allocates large amounts of memory when examining these files but I could successfully run tiffinfo against these where I couldn't run the info tool.

I noticed that someone else filed a similar bug against libtiff last year with no resolution yet. It's not obvious that this is a bug but it could be a DoS vector if ome-files-cpp backs a web application that accepts user data.

Here is a set of examples. Each of these is derived from the single-channel.ome.tif from the OME-TIFF sample data and is about 70k; they can lead to memory allocations of 8GB+:
examples.zip