gridtools/gtc

A stencil DSL toolchain written in Python.

gtc's Issues

Change GTC repository layout

Change the layout to have one top-level folder per sub-project.

Example:

REPO/
    gtc/
        src/
            gtc/
        docs/
        tests/
        setup.cfg
        setup.py
        pyproject.toml
        ...

    eve/
        src/
            eve/
        docs/
        tests/
        setup.cfg
        setup.py
        pyproject.toml
        ...

    gt_frontend/
        src/
        docs/
        tests/
        setup.cfg
        setup.py
        pyproject.toml
        ...

    cpputil/  # or gridtools-mesh or any other name
        cmake/
        include/
        tests/
        CMakeLists.txt
        ...

Add visitor utilities for simple tree traversals and filtering

• Add simple functions or classes to iterate over all the nodes of a tree in predefined orders
• Add filtering utilities based on iterators

Serialization

Prototype frontend for unstructured

Write an absolute minimal frontend to parse GTScript into GTIR

Create Pass infrastructure to handle node contexts

• Add node contexts
• Add symbol tables and scopes

Make simple MLIR example for parsing experiments

use e.g. the "loop" dialect from MLIR as an example

Allow type variables in Nodes

It is sometimes useful to restrict types of fields of fields of a node.

Objectives

• Reduce the amount of verification boilerplate code inside the translators.
• Reduce the amount of state to be passed to child nodes inside the translators
• Elimination of redundant nodes

Pydantic supports this out of the box (see pydantic/pydantic#556 and pydantic/pydantic#595).

Open questions:

1. How to check for a field type to be a subtype of generic type? This will be a common question if we allow this, but Python does not allow subclass checks with generic, i.e. the following does not work:

from typing import Generic, TypeVar, List
# raises: TypeError: Subscripted generics cannot be used with class and instance checks
issubclass(List[int], List[T])

2. Consequences of changing the Node base class to GenericModel
3. Is this compatible with mypy?

Example 1

T = TypeVar('T')
class Constant(GT4PyAstNode, Generic[T]):
    value: T

class ExampleNode:
    some_int_constant: Constant[Int]

Example 2

T = TypeVar('T')

class Stencil(GTScriptAstNode, Generic[T]):
    iteration_spec: List[Union[IterationOrder, LocationSpecification, Interval]]
    body: List[T]

class Computation(GTScriptAstNode):
    name: str
    arguments: List[Argument]
    stencils: List[Union[Stencil[Stencil[Statement]], Stencil[Statement]]]

Make frozen nodes hashable

SIR: transformation infer location type

Add customization of visit_ methods lookup order in eve visitors

Add a class toggle to enable or disable the use of node superclasses when looking up for suitable visit_ methods

Update code generator and test it with no mesh concept

Since lookup tables are passed by the user, and they cannot be generally generated, it makes little sense to have a concept for the mesh, and we can just take the mesh as parameter, as we do for the fields. We could treat mesh/neighbor tables as "state" of the computation.

new_infrastructure: Nodes with same field are not distinguished

If I have

class FieldAccess(Expr):
    name: Str

class VarAccess(Expr):
    name: Str

the visitor will not distinguish the 2 nodes. It will enter visit_FieldAccess even for VarAccess. This can be solved by type annotating the visit method's node argument.

However, no such workaround exists for the templates in the TemplatedGenerator. Adding an extra field in one of the 2 classes works, e.g.

class VarAccess(Expr):
    name: Str
    dummy: Optional[str]

Add FVM dependencies to gitpod

Cartesian: SIR, GTGenIR, KSwapIR

• Extent SIR for structured grids
• Implement a GTGenIR (an IR close to GridTools code)
• Implement an IR which allows optimization which are not compatible with GridTools

Change in behavior in "New infrastructure"

In the new_infrastructure branch there is a change in behavior in the code generator.

For a field like
a_field: Optional[List[SomeNode]], if optional is None, the code generator will now expand to N o n e instead of skipping over it. In the template I used ''.join(a_field)

Caching "derived information"

Just as a reminder to discuss it at some point...

Dawn has the concept of derived information which is some extra information computed from the IR and re-used by several transformations. (Things that are expensive to recompute.) Would be interesting to discuss if we could implement such a thing with automatic invalidation.

One example could be dependency graphs.
You would give_me_dependency(some_expression) and it will retrieve it from cache or compute it and store in cache.

This would be only a compiler performance optimization which we shouldn't do now.

Codegen: iteration index in reduction with sparse field is wrong

Mypy checks with enums

I create this issue to reference from gt4py comments.

@enum.unique
class MyEnum(eve.IntEnum):
    member: 0

reveal(MyEnum.member) # <- int but should be MyEnum

Run this in outside of the eve package (maybe problem also inside?).

@egparedes and I couldn't figure out what's going on, in a short debugging session, but we stopped as it should go away in the next refactoring.

Infer location type of local var as 2 passes

Enums are not typesafe

In my unstructured sir -> naive example I can assign sir.LocationType to a field which requires a naive.LocationType.

Transform unstructured GTIR into a definition IR

Mesh traits and concept helpers

• is_connectivity

• maybe has_connectivities (has_iterationspace)

• type for max_neighbors etc

• move to mesh.hpp and define LocationType tags (namespace topology)

Namespace in visit_ methods

Currently there is no way to distinguish Nodes with the same name from a different dialect.

Pass-Manager

Use SID bounds in max neighbors

• propagate static information
• provide a hook to provide static max neighbors in case of atlas where this info is runtim

SIR: parallel model declarations

Re-enable mypy in pre-commit

In #49 I disabled mypy again as there was an error I didn't know how to fix.

Validate model from assignment

Currently we only validate a Node on construction, but not if a field is assigned.

For validators this can be enabled in pydantic's config by setting

validate_assignment = True

However it seems this doesn't apply to root_validators. Bug in pydantic?

GTIR->NIR->UGPU->CUDA code

Finish lowering from GTIR to cuda code generation.