checker falsely flags halo inside shape

[bioimage-io]

checker falsely flags halo inside shape

[constantinpape]

Ok, I agree. This is an error in the spec (but note that it was an error from the beginning....). I will add a PR to fix it in there properly.
I think the fundamental problem is that we allow too many different ways of specifying the shape, which change the type of the shape field. This is not good design, but it's probably not worth reopening this can of worms now.

[constantinpape]

Yes, it looks like halo was mistakenly put here
instead of the OutputShape.
cc @FynnBe

[FynnBe]

I think the error lies in bioimage-io/spec-bioimage-io#76

In our "reference example" halo is also part of the output tensor description, not of the shape:
https://github.com/bioimage-io/pytorch-bioimage-io/blob/113e6d01989e39cd955cb2ead311086548d95026/specs/models/unet2d_nuclei_broad/UNet2DNucleiBroad.model.yaml#L45

This is because the shape might be defined with reference_input, etc. but it may also be a fixed shape : [1, 2, 3], and halo may be specified independently...
I see some arguments for including halo under shape, but as of now that is indeed 'wrong'

[constantinpape]

I think the error lies in bioimage-io/spec-bioimage-io#76

In our "reference example" halo is also part of the output tensor description, not of the shape:
https://github.com/bioimage-io/pytorch-bioimage-io/blob/113e6d01989e39cd955cb2ead311086548d95026/specs/models/unet2d_nuclei_broad/UNet2DNucleiBroad.model.yaml#L45

I am pretty sure that we have put halo in the shape (going back to the discussions about how to compute the output shape).
This is rather an error here and in the reference example.

[constantinpape]

This also becomes clear if you actually read what we have in the spec description:

- reference_input name of the reference input tensor
- scale list of factors 'output_pix/input_pix' for each dimension
- offset position of origin wrt to input
- halo the halo to crop from the output tensor (for example to crop away boundary efects or for tiling). The halo should be croped from both sides, i.e. shape_after_crop = shape - 2 * halo. The halo is not cropped by the bioimage.io model, but is left to be cropped by the consumer software. Use offset if the model output itself is cropped.

This refers to offset without any further modifiers, which strongly implies that its on the same level of hierarchy.

[FynnBe]

I see how this ambiguity arose over time, as we had further discussions about the meaning of halo and offset. While offset only makes sense with the presence of reference_input, halo is independent of how the output shape is defined. If at any time we decided to move halo into shape I am not opposed to changing the validator accordingly, but I am puzzled about the use case of a fixed output shape that has a halo. Because of this case I see no other way (without further changes) to keep halo out of shape.

[constantinpape]

If at any time we decided to move halo into shape

It was always part of shape, see these commits were we introduced the shape descriptions:
bioimage-io/spec-bioimage-io@9d1dfb6
bioimage-io/spec-bioimage-io@dc54550

I am puzzled about the use case of a fixed output shape that has a halo

This is again going back to the discussions we had about this, but halo is a hint for the consumer software on what parts of the output to crop away in order to get the "valid" part of the prediction, e.g. to avoid border effects.
So in that sense it should not matter if the output shape is fixed or computed from offset and scale, the treatment of halo is always identical.

[FynnBe]

So in that sense it should not matter if the output shape is fixed or computed from offset and scale, the treatment of halo is always identical.

exactly, but in case shape is specified explicitly, halo cannot be below shape like offset and scale

I am thinking of this scenario:

output:
  shape:  [10, 20, 30]  # explicit shape
  halo: [1, 2, 3]  # halo is outside of shape

[constantinpape]

See bioimage-io/spec-bioimage-io#77.

[constantinpape]

This is fixed.