Required Arguments to Herd Function

[hunhoffe]

Required Arguments to Herd Function

[hunhoffe]

Some actions items from this list that I think make sense after further discussion:

• Sizes arguments for segments and herds should probably be validated to be 2 dimensional (e.g., segment size [1] should probably not be valid)
• To clean up air python code, I should figure out how to hide the four arguments for a herd of default [1,1] size, the same way it is for segments.

[jgmelber]

Adding here for "documentation":

Multiple herds can exist in a segment and are placed to fit and the segment must have enough resources for that placement, the herds are guaranteed to run/exist at the same time within the spatial fabric the segment reserves. Same goes for segments within a launch, they are guaranteed to exist on the device at the same time or the launch cannot be executed. Therefore both herds and segments should have a shape. Though the shape of a segment is much more physical and must account for the structure of a target device's dataflow units (DUs), where a DU is the smallest unit of reservation/configuration. Launch has an "iteration space" for invocations of that launch and those can exist in space or time. It is up for debate whether the iteration space is a simple count or a shape.

[hunhoffe]

Here are a few points from discussion today:

• It's a bit unclear whether herd sizes correspond to physical blocks/shapes, or to iteration spaces
• Segments correspond to iteration spaces, so how segments function makes sense with [1] being valid (to be specific, 1 or 2 or 3 dimensions should be valid)
• Regardless, for a user, being able to specify the size of the herd in a multi-dimensional way and then get access to indices for each iteration may save the programmer from repetitive modular arithmetic to calculate indices. So we don't want to move to a single-dimension count argument as a matter of utility/convenience, regardless of what physical dimensions the device may have.

From this, the favored solution would be:

• Fix bugs/current state so that herds function the same way that segments do as shown in the examples in the initial issue writeup above
• Eventually, there should be an additional argument to herd declaration that specifies whether the size should correspond to the physical shape or not, and this flag should be percolated to the compiler passes which do the physical mapping to let the passes know if they are permitted to change the shape or not.

Outstanding implementation question: where should validation happen?

• My thought: If the herd op should then be able to take in a shape of 1, 2, (or 3 even) dimensions - it would be up to the compiler to decide, if the shape is meant to be taken as a physical description, whether the dimension is valid for the target device (e.g., for npu, I think a two dimensional physical shape would be required).