iydon/of.yaml

Python Interface to OpenFOAM Case (Configured Using YAML)

🔵⬜🟦⬜🟦
⬜⬜🟦⬜🟦
🟦⬜🟦🟦🟦
🟦⬜⬜⬜🟦
🟦⬜🟦🟦🟦

Iydon (梁钰栋 in Chinese)

· · · ·

GitHub Statistics

Code Statistics

===============================================================================
 Language            Files        Lines         Code     Comments       Blanks
===============================================================================
 Autoconf                9           45           36            0            9
 BASH                   32           98           34           32           32
 Batch                   7          343          289            3           51
 C                      36        4,418        2,599        1,060          759
 C Header               47        3,554        2,507          485          562
 CMake                   1           21           14            0            7
 C++                    13          611          462           40          109
 CSS                     6          493          480           12            1
 Cython                  1           16            7            5            4
 Dockerfile              3          215           95          100           20
 Forge Config            2           38           38            0            0
 FORTRAN Modern          2          201          164            0           37
 Go                      2          140          118            0           22
 JavaScript              2            4            4            0            0
 JSON                   40       21,830       21,828            0            2
 Julia                   2          120          105            0           15
 M4                      2        1,069          927            0          142
 MATLAB                155        6,266        4,650        1,017          599
 Makefile               69        1,272          888           85          299
 Python                836       86,430       74,424        2,102        9,904
 R                       5          256          215            6           35
 ReStructuredText       21        1,046          705            0          341
 Shell                  11          174          101           40           33
 SQL                    19        7,774        7,623           88           63
 SVG                     6          968          883            1           84
 TeX Class              26       14,113       10,567        2,414        1,132
 TeX Definition          3          207          195           12            0
 TeX                   676      328,353      179,029        3,421      145,903
 Plain Text             67      183,640            0      181,202        2,438
 TOML                   18          609          529           15           65
 XML                     1            7            7            0            0
 YAML                   94       37,690       35,842        1,147          701
-------------------------------------------------------------------------------
 HTML                   26        2,542        2,164          322           56
 |- CSS                  5          116          115            0            1
 |- JavaScript           9          461          396           11           54
 (Total)                          3,119        2,675          333          111
-------------------------------------------------------------------------------
 Jupyter Notebooks      11            0            0            0            0
 |- Markdown             7          288            0          259           29
 |- Python              10          370          324           11           35
 (Total)                            658          324          270           64
-------------------------------------------------------------------------------
 Markdown              381       26,913            0       20,808        6,105
 |- C                    1            7            6            0            1
 |- C++                  2           78           63            2           13
 |- HTML                 2           66           66            0            0
 |- JSON                 2        1,096        1,096            0            0
 |- MATLAB               1           62           28           29            5
 |- Markdown             1           51            0           50            1
 |- Python              35        1,965        1,569           82          314
 |- R                    2          214          178           36            0
 |- Rust                 2          269          237            6           26
 |- Shell               23        1,027          979            0           48
 |- SQL                  1            2            2            0            0
 |- TeX                  2           42           35            0            7
 |- XML                  1            8            8            0            0
 |- YAML                 7          263          234           17           12
 (Total)                         32,063        4,501       21,030        6,532
-------------------------------------------------------------------------------
 Rust                  191       20,121       17,145          694        2,282
 |- Markdown            45        8,067           58        5,930        2,079
 (Total)                         28,188       17,203        6,624        4,361
===============================================================================
 Total               2,823      751,597      364,674      215,111      171,812
===============================================================================
  

of.yaml's Issues

Sort out the import relationships in `__init__.py`

`Alltest` filtering test cases by OpenFOAM version

publish the package to pypi.org

https://pypi.org

Add identifier: vertical line

of.yaml/foam/base/parse.py

Line 25 in be61e9d

if any(c in key for c in '()*'): # (U|k|epsilon) -> "(U|k|epsilon)"

'()*|'

Materials related to remote operation

Related Libraries

• paramiko
• fabric

Ancient Code

import os
import pathlib
import stat
import subprocess
import textwrap

import paramiko
import toml

from typing import Any, Callable, Dict, Iterator, Optional, Tuple, Union

from .servers import servers


PathLike = Union[str, pathlib.Path]
StandardIO = Tuple[3*(paramiko.channel.ChannelFile, )]


indent = lambda s: textwrap.indent(str(s), 4*' ')


class Config:
    '''配置信息
    '''
    NONE = 0  # None in toml
    TODO = ''  # todo
    data_fields = ('username', 'password', 'private_key_path')
    meta_fields = ('rsync', )

    def __init__(
        self,
        server: str, software: str, version: str,
        path: PathLike = 'config.toml', encoding: str = 'utf-8',
    ) -> None:
        assertion = lambda s, d: f'Accessible {s}(s): ' + ', '.join(d)
        self._keys = server, software, version
        assert (server:=servers.get(server)), assertion('server', servers)
        assert (software:=server.get(software)), assertion('software', server)
        assert (version:=software.get(version)), assertion('version', software)
        self._main = version  # server-software-version
        self._path = pathlib.Path(path)
        self._path.touch(exist_ok=True)
        self._encoding = encoding
        self._data = toml.loads(self._path.read_text(encoding))
        # 配置文件默认值
        self._data.setdefault('meta', {f: self.TODO for f in self.meta_fields})
        self._data \
            .setdefault(self._keys[0], dict()) \
            .setdefault(self._keys[1], dict()) \
            .setdefault(self._keys[2], {f: self.TODO for f in self.data_fields})

    def __getitem__(self, key: str) -> Optional[str]:
        value = self.data.get(key)
        return value if value!=self.NONE else None

    def __str__(self) -> str:
        destination = f'{self["username"]}@{self.host}:{self.port}'
        return f'<Config>{"::".join(self._keys)} {destination}</Config>'

    @property
    def data(self) -> Dict[str, Any]:
        # data = self._data
        # for key in self._keys:
        #     data = data[key]
        # return data
        server, software, version = self._keys
        return self._data[server][software][version]

    @property
    def meta(self) -> Dict[str, Any]:
        return self._data['meta']

    @property
    def host(self) -> str:
        return self._main.host

    @property
    def port(self) -> str:
        return self._main.port

    def input(
        self,
        input: Callable[[str], str] = input, pattern: str = '({}) >> ',
    ) -> 'Config':
        for d1ct in (self.meta, self.data):  # s1mple
            for key, value in d1ct.items():
                if value == self.TODO:
                    d1ct[key] = input(pattern.format(key)) or self.NONE
        return self.save()

    def save(self) -> 'Config':
        self._path.write_text(toml.dumps(self._data), self._encoding)
        return self


class Remote:
    '''远程连接
    '''
    def __init__(self, config: Config) -> None:
        self._config = config
        self._trans = paramiko.Transport((config._main.host, int(config._main.port)))
        self._ssh = self._sftp = None
        self._path = [pathlib.PureWindowsPath, pathlib.PurePosixPath][config._main.posix]
        # 后处理
        self._trans.connect(username=config['username'], **self._connect_kwargs())

    def __enter__(self) -> 'Remote':
        return self

    def __exit__(self, *_) -> None:  # type, value, traceback
        self.close()

    def __str__(self) -> str:
        return f'<Remote>\n{indent(self._config)}\n</Remote>'

    @classmethod
    def from_config(cls, *args, **kwargs) -> 'Remote':
        '''如果需指定 input 的参数，请使用完整构造函数

        - Example:
            >>> with Remote.from_config('sustech_iydon') as remote:
            ...     pass

            >>> with Remote(Config('sustech_iydon')) as remote:
            ...     pass
        '''
        return cls(Config(*args, **kwargs).input())

    @property
    def config(self) -> 'Config':
        return self._config

    @property
    def ssh(self) -> paramiko.SSHClient:
        if self._ssh is None:
            self._ssh = paramiko.SSHClient()
            self._ssh._transport = self._trans
            self._ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
        return self._ssh

    @property
    def sftp(self) -> paramiko.SFTPClient:
        if self._sftp is None:
            self._sftp = paramiko.SFTPClient.from_transport(self._trans)
        return self._sftp

    @property
    def Path(self) -> pathlib.Path:
        return self._path

    def close(self) -> None:
        self._ssh and self._ssh.close()  # is not None
        self._sftp and self._sftp.close()
        self._ssh = self._sftp = None

    def copy(
        self,
        local_path: PathLike, remote_path: PathLike, down: bool = True,
    ) -> None:
        '''拷贝文件

        - Argument:
            - down: local_path<-remote_path if down else local_path->remote_path
        '''
        method = getattr(self.sftp, 'get' if down else 'put')
        method(localpath=str(local_path), remotepath=str(remote_path))

    def exec(self, command: str, encoding: Optional[str] = None) -> Tuple[str, str]:
        '''ssh: exec command
        '''
        _, stdout, stderr = self._exec(command)
        encoding = encoding or self._config._encoding
        return stdout.read().decode(encoding), stderr.read().decode(encoding)

    def exists(self, remote_path_or_directory: PathLike) -> bool:
        '''远程文件或文件夹是否存在
        '''
        try:
            self.sftp.stat(str(remote_path_or_directory))
        except OSError:
            return False
        else:
            return True

    def mkdir(self, remote_directory: PathLike) -> None:
        '''新建远程文件夹

        - Note:
            - parents=True, exist_ok=True
        '''
        path = self._path(remote_directory) / 'NULL'
        for parent in reversed(path.parents):
            try:
                self.sftp.mkdir(str(parent))
            except OSError:
                pass

    def stat(self, remote_path_or_directory: PathLike) -> paramiko.SFTPAttributes:
        return self.sftp.stat(str(remote_path_or_directory))

    def sync(
        self,
        local_directory: PathLike, remote_directory: PathLike,
        down: bool = True, dry_run: bool = False, **_,  # verify
    ):
        '''rsync 命令
        '''
        local, remote = pathlib.Path(local_directory), self.Path(remote_directory)
        destination = f'{self._config["username"]}@{self._config.host}:{remote}'
        # parts of command
        rsync = self._config.meta['rsync'].format(port=self._config.port)
        options = ['--dry-run'] if dry_run else list()
        from_, to = (destination, local) if down else (local, destination)
        parts = (rsync, *options, from_, to)
        process = subprocess.run(' '.join(map(str, parts)), cwd=os.getcwd(), shell=True)
        assert process.returncode==0, repr(parts)

    def syncpy(
        self,
        local_directory: PathLike, remote_directory: PathLike,
        down: bool = True, dry_run: bool = False, verify: bool = True, **_,
    ) -> None:
        '''Python 实现的同步目录

        - Note:
            - 仅通过校验文件大小判断是否一致（可能后续考虑 mtime）

        - See Also
            - `Remote::copy`
        '''
        copy = (lambda l, r, d: print(l, '><'[d], r)) if dry_run else self.copy
        exists, mkdir, order = (
            lambda p: p.exists(), lambda d: d.mkdir(0o777, True, True), lambda *x: x,
        ) if down else (self.exists, self.mkdir, lambda *x: reversed(x))
        to_directory, from_directory = order(
            pathlib.Path(local_directory), self._path(remote_directory)
        )
        stat = lambda p, loc: p.stat() if loc else self.stat(p)
        # from -> to
        directory = to_directory / from_directory.name
        for from_path in self._walk(from_directory, not down):
            to_path = directory / from_path.relative_to(from_directory)
            not dry_run and mkdir(to_path.parent)
            if verify and exists(to_path):
                s1, s2 = stat(to_path, down), stat(from_path, not down)
                if s1.st_size==s2.st_size:  # s1.st_mtime==s2.st_mtime
                    continue
            copy(*order(to_path, from_path), down)

    def walk(self, root: PathLike) -> Iterator[pathlib.Path]:
        '''遍历服务器文件

        - Example:
            >>> for path in remote.walk(remote.Path(...)):
            ...     pass
        '''
        # assert stat.S_ISDIR(self.sftp.stat(str(self._path(root))).st_mode)
        yield from self._walk(self._path(root), local=False)

    def _connect_kwargs(self) -> Dict[str, Any]:
        if (password:=self._config['password']) is not None:
            return {'password': password}
        if (path:=self._config['private_key_path']) is not None:
            return {'pkey': paramiko.RSAKey.from_private_key_file(path)}
        return dict()

    def _exec(self, command: str) -> StandardIO:
        '''底层 paramiko wrapper
        '''
        return self.ssh.exec_command(command)  # stdin, stdout, stderr

    def _sync_down(self) -> None:
        pass

    def _sync_up(self) -> None:
        pass

    def _walk(self, root: pathlib.Path, local: bool = True) -> Iterator[pathlib.Path]:
        if local:  # 有现成方法，避免递归
            for dirname, _, filenames in os.walk(root):
                yield from map(pathlib.Path(dirname).__truediv__, filenames)
        else:
            # assert isinstance(root, self._path)
            for attr in self.sftp.listdir_attr(str(root)):
                path = root / attr.filename
                yield from (
                    self._walk(path, local=False) if stat.S_ISDIR(attr.st_mode)
                    else (path, )
                )


class Scheduler:
    '''服务调度器
    '''
    def __init__(self, config: Config) -> None:
        self._remote = Remote(config)

    def __enter__(self) -> 'Scheduler':
        return self

    def __exit__(self, *_) -> None:
        self._remote.close()

    def __str__(self) -> str:
        return f'<Scheduler>\n{indent(self._remote)}\n</Scheduler>'

    @classmethod
    def from_config(cls, *args, **kwargs) -> 'Scheduler':
        '''
        - See Also:
            - `Remote::from_config`
        '''
        return cls(Config(*args, **kwargs).input())

    @property
    def config(self) -> 'Config':
        return self._remote._config

    @property
    def remote(self) -> 'Remote':
        return self._remote

add `Makefile`

• test
• mkdocs
• ...

Use the import statement directly, not the lib

of.yaml/foam/parse/static.py

Line 92 in c3e459c

with lib['py7zr'].SevenZipFile(in_, mode='r') as z:

    @match.register('embed', '7z')
    def _(self, static: Dict) -> None:
        import py7zr

        out = self._out(static['name'])
        with py7zr.SevenZipFile(io.BytesIO(static['data']), mode='r') as z:
            z.extractall(path=out.parent)

Use typing-extensions package

https://pypi.org/project/typing-extensions/

import typing_extensions as t

class Test:
    def test(self) -> t.Self:
        return self

YAML 添加 OpenFOAM 版本号

虽然目前参考的算例均来自 OpenFOAM-7，但是有如下添加版本号的原因：

• 方便 Alltest 时选用不同环境
• 记录算例的 meta 信息

如果能在 YAML 文件名上也区分版本号就更好了，这样一旦涉及到不同版本的算例时文件名不至于重复。

Improve format of the code

• Trim trailing whitespace
• Convert indentation to spaces
• Add space after comma and comment
• End the file with a newline
• ...

python3.7 compatibility issues

AttributeError: module 'functools' has no attribute 'cached_property'
AttributeError: module 'functools' has no attribute 'singledispatchmethod'

Move cases to a new repository

<repository>
├── README.md
├── OpenFOAM
│   ├── README.md
│   ├── 7
│   │   └── ...
│   ├── 9
│   │   └── ...
│   └── ...
├── third_party
│   ├── README.md
│   ├── BasicOpenFOAMProgrammingTutorials
│   │   └── ...
│   ├── openfoam_tutorials
│   │   └── ...
│   └── ...
└── ...

unittest!

List some OpenFOAM related projects

https://libs.garden/python/search?q=openfoam&sort=pushed

• https://github.com/jaheyns/CfdOF
• https://github.com/fluiddyn/fluidfoam
• https://github.com/DLR-RY/caseFoam
• https://github.com/greole/owls
• https://github.com/qingfengxia/Cfd
• https://github.com/ZhangYanTJU/fitData_Cantera
• https://github.com/ladybug-tools/butterfly
• https://github.com/Spationaute/SuperMarine
• https://github.com/takaakiaoki/ofblockmeshdicthelper
• https://github.com/Unofficial-Extend-Project-Mirror/openfoam-extend-Breeder-other-scripting-PyFoam
• https://github.com/nicolasedh/salomeToOpenFOAM
• https://github.com/xu-xianghua/ofpp
• https://github.com/openfoamtutorials/openfoam_python

regulate `README.md`

Refer to the template to regulate README.md

Add Chinese documentation

todo!

add `CHANGELOG`

• v0.3
  • add partial support for standard list
  • YAML file with multiple documents (meta, foam, static)
• v0.4
  • rewrite single file to package form
  • parse command line parameters with click
• v0.5
  • add version handling

The original data should not be modified

of.yaml/foam/parse.py

Line 52 in f7212cb

element.pop(head[0])

Typo: permision

of.yaml/foam/core.py

Line 130 in ca6281b

self._write(out, static['data'], static.get('permision', None))

embed binary data in YAML files as multiple documents

of.yaml/tutorials/multiphase/interFoam/laminar/sloshingTank3D6DoF.yaml

Lines 22 to 24 in 5ac987d

	name: gen6DoF
	type: 7z
	data: static/sloshingTank3D6DoF-gen6DoF.7z

---
meta:
    static:
        -
            name: gen6DoF
            type: embed/7z
            data: static/sloshingTank3D6DoF-gen6DoF.7z
---
gen6DoF: !!binary |
    N3q8ryccAAQNYUZhXwoAAAAAAABaAAAAAAAAACx8rOngGgAKV10ABQ0pPROkhrs7DAcooPWoEZTr
    l5BPTr5w8ZRQP6Jo5ffp6lYIiraB41qo6NwuCfBUHn5xarJflMZju3kb5t1mirtiegf3VxGG4Lkw
    pe8d2R6Nq2GqXiBLd/VK1HNQ7WhuvGwREVEuwa7UBUSbvMSGeR0qOeOUKwwn982zpLicZBqCWY3R
    wUtUMpN8XiPJuyWTTudA4ahMoNW9pSluy1k+P+3TvZb6Is3gJ3mMRq7ANsMkyKiDWsqKsGlBFRwH
    7HppNnXfsrtllyh2sbWqP254RZmT+ytz/X+IuVFGQyLaCMSlkjk70T/6drP/CsE1+ybvqlpduVdB
    q7HSRDxC1uP9rY039pjV/9YF4GlEe18qXN+CavsfU0JZUJUrALiBRo3Y68N1dG8ghHb3JIrvEuJ6
    QSTXigXyqB+uZWf3FRs4hZgf9aa/TVKFCfYGhGRCgB7AQTBOWHl44WsB46golZL4gZNOx0uu9cQX
    TECTVh57ZXy4fqq/cKZRHFGM1QRcvIN2KmTwblccZuwW3vsdpsC8OBdS5eB1xDVpgvCHawHtXcBi
    g2cfzHUx/IuJ3i6qu9yLPIA+AtQ/vVl2KbhE1cdoYAlNbOZcXYAdhKcVwOt2XZz1xlHsjkeVntjs
    SWxjdsQZ+/gvHwV5yS3lW2PoRqCfaIqIrDaCWU2ayPw9QQK2hgIDKBsLQ2ufHPZFkbq9mnxgagz4
    Hd5Km8sc/1HuG6HZVHCGs0Y4f7ikFO+rqUOkSe6q6zg+oytkkmsye2RTWwFitZ570781LsOq2gZ8
    3C48P6YoVf59BXtGPAzR1NY2g87Y9lxFabecpwSPFdHGkJ0Lc15sfVlAoF3/fIUXvvzuKr7QkPNQ
    jxB7emOeSoMlmS4w7XfOKUegpKFdonuq+hY2VTcs8ty1dGJXZcr/FWHDItj8igLmmWj0+2qzvQqJ
    AJVqcTg2i0q9Jhqnmcc7PQdWr1b9vI8V2LgHEfxuzl7yRaXIIF8IwAH5DVUaMZUoUF366NSf+m3R
    h/zDU80T2VibylgG252tsUTBbMqNB9T7GWiEBj6y6ugPkLfzziQbo2fqU5rVnGORYJvnyBgfJ1DK
    aei5l7tDZGAHXuDT2Y64QDMlopOB2CZ+W4p59pjZWiBhlBwwv9F7zQKoTPjVH9utYwBEwVSrS3KL
    GmYQW3rbx0guChBOZijqkyw2iUuDiGzwYguP/Et3ild4sSEYa6rhq0V3DKWH6g2lGs3/woBryKa2
    2PM92SWDg2qhD97UV7+8ctIe+eE2sFhqMtIjeF8tVmyh6WiSilPZ8YALRX+42+8QW8ky6N6WzrRr
    fB+5i1X37DMUzfB77/vh6PYYaPN/wfgwjFmXq079rMay+azu4ZecTjuM7PWZgGidnsMq/9gEDGZi
    DBG0atIQ81oivTPSLnioyhwThGQyh0nL4xXx6/0hsifqvZFIdJ1OHspf8iRr9YVG7xkRH7hHzPbS
    75+9EN00hTMqv2J6+1JM/Q/xZuSUvOF3gcaGT0XF5BSp9Fyna461MceqzLIG/zZme55fYtFQSsw5
    RAZQeOk4xUoMpI2quiyoRsZZjiMV5NofD9URc53HCpiLmC6WWJzjSF1l3gEf9rMAQ8h/pg2QW5y2
    ixb1HFxm3tWWuqtp7CPDLSYAYPiijGuTFm6JQrz1XV5RaJ+brDj08q0zpS3Qaa8vfGXbhO9GM8qV
    IGIVXvss2ExZMY54CeCqeKMQ0POQDQm3f+pFZJqITWyNr8fzjtRA40SPUxww8+OhXptwUbnPD6k3
    1dy8C8iVswoPka4Isyoa01Tzqo0t1nS3pzAd8wdg9R+cL2X0XSQxXTH7l9Q7of2n+q+aidWS8hrC
    DGzTy+YAiI4K0qzCMXBIwFG6GJGiX425huZ/r8wAbu3p592XsGeC0C5spqu4mRjA15imL+ng+p3s
    j21V0gdheISai2fyZtK666EbJIvu8qNnYd3UT5rBfmFl3k5siPviBsuHXPKDJ9H88jWBq52bXl+V
    Zuo1LGHI3F74VotkWtslfSJVQB+0oX9QEPXFcvtDFBszEW3zwOUJXNCGBQIFxvwnKYqyGiI1yGIU
    vtXh8NfQp04Ennpi+SXP9q/mVFu/loG9f+7uZOHVzbBlivaK5UzKD7531ZzHrIOycZxWb16AnisU
    CiZXeIDYXUhyO8pzyP8Qx2wHMXX5F3AOfP1PpmhMY+rdWwd0Qlmi49wWLRCzyeVSDD8gY4VPpBrs
    FNFvfOxEWneTzEjAm8XiTIg5RNYMsHxqxwb8CEySrYIATiwD6IBiwFFD0Id0bLehBsnoBsCRpSws
    dayVVgWVTdeyMZV4F6NnbjBwb++wDaVuMlwc9JRln0jolK3OWSR8WtqxdrjCG7jw514JDoOi9/js
    8I68T951x0hBDduVQ/chIwVkEcKnpw0rrQ/hgZahAJLcdDV72WvkQN+FPASim7D7mwsQPinECN6S
    dV6jdD5ZoNhJVVzjk2ILyc8HSp+PfIUPjbrd8OHf5zuZVSM7lB2D7pYjGkkocL7PIqi/L/JUJy/j
    NHySWdheskgT4Fc7LJyoXfu6LqxQY1nvpCOZipYrX2X1JJfgVXxqu5xiS4icJ1pmtblhrb4Xjgul
    3YPOIEK7CrZ9UePvK4ReUmdRjPeos0pXIEGF2EUqrg4g5ZmwsBQ84w6ufoOyd/MOPZ2DYPx4vck8
    Yv7RWMeBefq2gEMWGwdPdmaNdUuJam6Xu7TLzKKkKuB15uUlgHujj96PZJd3c0GfCD+i+j2Nf9LB
    4ro8sX8j330mnzP/VB0wZIXeAwCAgKba/q1GKalb7bL1DxyHVV2Og4jZkOaJ4Yyxh7LOlTS7IOM+
    0FwpQuJGJ4n/8FeAO+z5LPkFHx/E6ZpKGl2kWkce+Km1+1Rt7CnEL+AIca5rvmO4TqIziQoRMcJe
    dC2IZKAKpFc0mShQ5JgaKMPb2B3CJJdysEi/TOtPZOwjk/LJ9fwf4GupxvMZ2ZIIccsC5IGCQHCt
    54LxGm97oFcwMrH3EVvhcKPfSCkKcBIrcHeXOIranttQlMMiZ+dB53Ojxahs/8C+RigIKUoTi4vt
    3rj95P/inoN5vKbwu9qRNhqXszNvhAwd8ws1qF9uHKQJM+R7ExPJu5muoPJep75sYDXsHzPRVaYZ
    HZMyg5Xg85fQ8q9zQGAO0JOXbhsaENS1lxFBJW44I9+A4WKTIzCRf28e0opO1a1f6oOoPJQ04/du
    jEWE5mw2JRuENl++xo1jp9leInKNNx1ebd+oBqjqAGxc5/ZHUkKpGMPuC14GsGnOn1ORFxzzVyBE
    kM8s0BYohhnrL/vXk6N7b4XkCKtwzmqM8VztIkABbkkyw58//YZ6t2naxPO2LRkd8GoMOuyeIToF
    EDjfQHZYZGsmC53Bf0fvVl+EdUc++zJNt04ss3FKCH891R+hCeAWHhpHqLNLMjSgdIq9GL7ScPSq
    CG+5i47aqN78SkRfeUz/NT8LX7xdCcR6m0kK0VBcHmcvwYhg/eR0OoBuOHR7hWGFn+oljorPVdNP
    Jt4GbvJLAAABBAYAAQmKXwAHCwEAASEhAQIMmgEACAoBeHU+yAAABQEZCgAAAAAAAAAAAAAREwA2
    AEQAbwBGAC4AZABhAHQAAAAZABQKAQDLI/16ZAbYARUGAQAgAAAAAAA=
---

Also consider replacing type path with path/text, 7z with path/7z

add `Alltest`

Project uses automated testing instead of manual testing

move `dictToFoam.py` to `dictToFoam/__main__.py`

Should we move dictToFoam.py to dictToFoam/__main__.py to reduce complexity in the root directory?

use list for openfoam versions

of.yaml/tutorials/incompressible/simpleFoam/airFoil2D.yaml

Line 2 in 2ff2161

openfoam: 7

openfoam: [7]

[standalone] "Foam" is not defined

of.yaml/script/standalone.py

Line 36 in efd99e1

code = pattern_im.sub('', obj.replace(': Foam', ': \'Foam\''))

of.yaml/foam/app/information/core.py

Line 20 in efd99e1

def __init__(self, foam: t.Optional[Foam]) -> None:

Put loader and dumper together

Code from ihpc:

__all__ = ['dumper', 'loader']


import json
import pathlib as p
import pickle

from ..base.lib import tomlkit, yaml
from ..base.type import Document, Path


class dumper:
    '''Dumper for json, pickle, toml, yaml'''

    _mapper = {
        ('json', ): lambda doc: json.dumps(doc).encode(),
        ('pickle', 'pkl'): lambda doc: pickle.dumps(doc),
        ('toml', ): lambda doc: tomlkit.dumps(doc).encode(),
        ('yaml', 'yml'): lambda doc: yaml.dump(doc).encode(),
    }

    @classmethod
    def to_bytes(cls, document: Document, type: str = 'json') -> bytes:
        for types, dumps in cls._mapper.items():
            if type in types:
                return dumps(document)
        raise Exception(f'"{type}" is not a valid type string')

    @classmethod
    def to_string(cls, document: Document, type: str = 'json') -> str:
        return cls.to_bytes(document, type).decode()

    @classmethod
    def to_path(cls, document: Document, path: Path) -> p.Path:
        path = p.Path(path)
        path.write_bytes(cls.to_bytes(document, path.suffix[1:]))
        return path


class loader:
    '''Loader for json, pickle, toml, yaml'''

    _mapper = {
        ('json', ): json.loads,
        ('pickle', 'pkl'): pickle.loads,
        ('toml', ): tomlkit.loads,
        ('yaml', 'yml'): yaml.load,
    }

    @classmethod
    def auto_from_bytes(cls, content: bytes) -> Document:
        for loads in cls._mapper.values():
            try:
                return loads(content)
            except Exception:
                pass
        raise Exception(f'Unable to recognize content type')

    @classmethod
    def auto_from_string(cls, content: str) -> Document:
        return cls.auto_from_bytes(content.encode())

    @classmethod
    def from_bytes(cls, content: bytes, type: str = 'json') -> Document:
        for types, loads in cls._mapper.items():
            if type in types:
                return loads(content)
        raise Exception(f'"{type}" is not a valid type string')

    @classmethod
    def from_string(cls, content: str, type: str = 'json') -> Document:
        return cls.from_bytes(content.encode(), type)

    @classmethod
    def from_path(cls, path: Path) -> Document:
        path = p.Path(path)
        return cls.from_bytes(path.read_bytes(), path.suffix[1:])

module 'urllib' has no attribute 'request'

of.yaml/foam/base/core.py

Line 96 in 01e6bf3

with urllib.request.urlopen(url) as f:

Migrate properties to Foam class

Because there is no need to save to get these properties.

of.yaml/foam/command/core.py

Lines 48 to 57 in 03db1fe

	@f.cached_property
	def application(self) -> str:
	return self._foam['foam']['system', 'controlDict', 'application']
	@f.cached_property
	def number_of_processors(self) -> int:
	try:
	return self._foam['foam']['system', 'decomposeParDict', 'numberOfSubdomains']
	except:
	return 1

Move `dest` parameter to this function

of.yaml/dictToFoam.py

Line 51 in d93e02b

def save(self) -> None:

def save(self, dest: p.Path) -> None:
    ...

RecursionError: maximum recursion depth exceeded in comparison

of.yaml/foam/__init__.py

Line 9 in 53c787b

_shutil.copytree = _compat.shutil.copytree

Version tags are too messy

不影响兼容性的小修改只改变小版本号，例如从 0.7.1 跳至 0.7.2，等到最后出现影响兼容性的大修改时，先添加 v0.7 的标签，预示 0.7.* 版本已经结束，再将版本号跳至 0.8.0。以此类推。

至于 https://github.com/iydon/of.yaml/blob/main/CHANGELOG.md 的修改，在没有标签前不为版本添加标签超链接。

Add support for VTK post-processing

https://pypi.org/project/vtk

PEP-0560: Core support for typing module and generic types

of.yaml/foam/base/type.py

Lines 18 to 28 in 81ada0b

	def Array(*dimensions: int) -> t.Union['np.number', 'np.ndarray']:
	assert dimensions and all(d>=0 for d in dimensions)
	if t.TYPE_CHECKING:
	dimensions = set(dimensions)
	if dimensions == {0}:
	return lib['numpy'].number
	else:
	return lib['numpy'].ndarray
	else:
	return t.Any

https://peps.python.org/pep-0560/

class Array:
    def __class_getitem__(cls, dimensions: t.Tuple[int, ...]) -> type:
        ...

Dynamic load third-party libraries

import typing as t

class Foam:
    @property
    def py7zr(self) -> t.Optional[t.Any]:
        try:
            import py7zr
        except ImportError:
            py7zr = None
        return py7zr

Use empty dictionary instead of `"": ""`

Code

of.yaml/tutorials/IO/fileHandler.yaml

Lines 297 to 298 in c3bd34d

	cloudFunctions:
	"": ""

cloudFunctions: {}

Reference

• Syntax for empty dictionary in YAML

multiphase/multiphaseInterFoam/laminar/mixerVessel2D incorrect FoamFile

of.yaml/tutorials/multiphase/multiphaseInterFoam/laminar/mixerVessel2D.yaml

Lines 116 to 119 in a1a6742

	MRFProperties:
	FoamFile:
	class: MRFProperties
	object: g

https://github.com/OpenFOAM/OpenFOAM-7/blob/63349425784a38457964e6470087a2eaac8f0673/tutorials/multiphase/multiphaseInterFoam/laminar/mixerVessel2D/constant/MRFProperties#L8-L15

Document is out of date

• Architecture
• Cases
• ...

use mkdocs to write documentation

KeyError: 'te.Self'

of.yaml/foam/util/private/case.py

Lines 37 to 38 in e9ddf29

	def required(self, t: float, dt: float):
	return self.dict_without_keys(vars(), 'te.self')

def required(self, t: float, dt: float):
    return self.dict_without_keys(vars(), 'self')

Unstructured grids have different grid densities

https://stackoverflow.com/questions/66891594/calculate-the-centroid-of-a-3d-mesh-of-triangles

of.yaml/foam/postprocessing/core.py

Line 85 in 03db1fe

def centroid(self, key: str) -> 'np.ndarray':

LICENSE.txt appears in the site-packages directory

of.yaml/pyproject.toml

Line 12 in 34daf05

include = ["LICENSE.txt"]

$ ls -1 ~/.local/lib/python3.7/site-packages/

LICENSE.txt
foam
ifoam-0.11.7.dist-info
...

Direct parsing of OpenFOAM case

Code

foam = Foam.from_foam(...)

Reference

• https://doc.cfd.direct/openfoam/user-guide-v9/basic-file-format
• https://github.com/lark-parser/lark
• https://www.openfoam.com/documentation/user-guide/2-openfoam-cases/2-2-basic-inputoutput-file-format

cp: cannot stat '../../processor*': No such file or directory

of.yaml/tutorials/IO/fileHandler.yaml

Line 332 in e66ea65

cp -r ../../processor[0-1] . && \

'Data' object has no attribute 'get'

of.yaml/foam/command/core.py

Line 44 in 4c55131

pipe = (self._foam['other'] or {}).get('pipeline', None)

Every change before release should be tested, don't take it for granted!

Use mypy to check static types

• 🛑 foam/base/lib.py: Library stubs not installed for "yaml" (or incompatible with Python 3.8)
• 🛑 foam/base/core.py: Value of type "Optional[Data]" is not indexable
• 🛑 foam/base/core.py: Value of type "Optional[Data]" is not indexable
• 🛑 foam/base/core.py: No overload variant of "get" of "dict" matches argument types "str", "List[<nothing>]"
• 🛑 foam/base/core.py: Value of type "Optional[Data]" is not indexable
• 🛑 foam/base/core.py: Value of type "Optional[Data]" is not indexable
• 🛑 foam/base/core.py: Argument 3 to "copytree" has incompatible type "**Dict[str, bool]"; expected "Union[None, Callable[[str, List[str]], Iterable[str]], Callable[[Union[str, PathLike[str]], List[str]], Iterable[str]]]"
• 🛑 foam/base/core.py: Argument 3 to "copytree" has incompatible type "**Dict[str, bool]"; expected "Callable[[str, str], None]"
• 🛑 foam/base/core.py: Argument 1 to "_extract_files" of "Foam" has incompatible type "Union[Dict[str, Any], List[Dict[str, Any]]]"; expected "Dict[str, Any]"
• 🛑 foam/base/core.py: Item "None" of "Optional[Any]" has no attribute "setdefault"
• 🛑 foam/base/core.py: Unsupported left operand type for / ("None")
• 🛑 foam/app/postprocess/core.py: Unsupported left operand type for / ("None")
• 🛑 foam/app/postprocess/core.py: "Array" expects no type arguments, but 2 given
• 🛑 foam/app/postprocess/core.py: Invalid type: try using Literal[1] instead?
• 🛑 foam/app/postprocess/core.py: Invalid type: try using Literal[2] instead?
• 🛑 foam/app/postprocess/core.py: "Array" expects no type arguments, but 2 given
• 🛑 foam/app/postprocess/core.py: Invalid type: try using Literal[1] instead?
• 🛑 foam/app/postprocess/core.py: Invalid type: try using Literal[2] instead?
• 🛑 foam/app/postprocess/core.py: "Array" expects no type arguments, but 2 given
• 🛑 foam/app/postprocess/core.py: Invalid type: try using Literal[0] instead?
• 🛑 foam/app/postprocess/core.py: Invalid type: try using Literal[1] instead?
• 🛑 foam/app/postprocess/core.py: Argument 1 to "probes" of "PostProcess" has incompatible type "Tuple[float, ...]"; expected "Tuple[float, float, float]"
• 🛑 foam/app/postprocess/core.py: Invalid index type "Tuple[float, ...]" for "Dict[Tuple[float, float, float], Dict[str, Dict[float, Array]]]"; expected type "Tuple[float, float, float]"
• 🛑 foam/app/postprocess/core.py: "Array" expects no type arguments, but 2 given
• 🛑 foam/app/postprocess/core.py: Invalid type: try using Literal[0] instead?
• 🛑 foam/app/postprocess/core.py: Invalid type: try using Literal[1] instead?
• 🛑 foam/app/postprocess/core.py: Need type annotation for "ans" (hint: "ans: Dict[<type>, <type>] = ...")
• 🛑 foam/app/postprocess/core.py: No overload variant of "set" matches argument type "Array"
• 🛑 foam/app/postprocess/core.py: Value of type "Array" is not indexable
• 🛑 foam/app/postprocess/core.py: "Array" expects no type arguments, but 1 given
• 🛑 foam/app/postprocess/core.py: Invalid type: try using Literal[2] instead?
• 🛑 foam/app/postprocess/core.py: "Array" expects no type arguments, but 1 given
• 🛑 foam/app/postprocess/core.py: Invalid type: try using Literal[2] instead?
• 🛑 foam/app/postprocess/core.py: "Array" expects no type arguments, but 2 given
• 🛑 foam/app/postprocess/core.py: Invalid type: try using Literal[1] instead?
• 🛑 foam/app/postprocess/core.py: Invalid type: try using Literal[2] instead?
• 🛑 foam/app/postprocess/core.py: "Array" expects no type arguments, but 2 given
• 🛑 foam/app/postprocess/core.py: Invalid type: try using Literal[1] instead?
• 🛑 foam/app/postprocess/core.py: Invalid type: try using Literal[2] instead?
• 🛑 foam/app/postprocess/core.py: "Array" expects no type arguments, but 1 given
• 🛑 foam/app/postprocess/core.py: Invalid type: try using Literal[1] instead?
• 🛑 foam/app/postprocess/core.py: Value of type "Array" is not indexable
• 🛑 foam/app/postprocess/core.py: "Array" expects no type arguments, but 1 given
• 🛑 foam/app/postprocess/core.py: Invalid type: try using Literal[1] instead?
• 🛑 foam/app/postprocess/core.py: Value of type "Array" is not indexable
• 🛑 foam/app/postprocess/core.py: "Array" expects no type arguments, but 1 given
• 🛑 foam/app/postprocess/core.py: Invalid type: try using Literal[1] instead?
• 🛑 foam/app/postprocess/core.py: Value of type "Array" is not indexable
• 🛑 foam/app/postprocess/core.py: "Array" expects no type arguments, but 2 given
• 🛑 foam/app/postprocess/core.py: Invalid type: try using Literal[1] instead?
• 🛑 foam/app/postprocess/core.py: Invalid type: try using Literal[2] instead?
• 🛑 foam/app/postprocess/core.py: "Array" has no attribute "ndim"
• 🛑 foam/app/postprocess/core.py: Value of type "Array" is not indexable
• 🛑 foam/app/postprocess/core.py: Unsupported left operand type for * ("Array")
• 🛑 foam/app/postprocess/core.py: "Array" has no attribute "ndim"
• 🛑 foam/app/postprocess/core.py: Item "Array" of "Optional[Array]" has no attribute "T"
• 🛑 foam/app/postprocess/core.py: Item "None" of "Optional[Array]" has no attribute "T"
• 🛑 foam/app/postprocess/core.py: No overload variant of "sum" matches argument type "Array"
• 🛑 foam/app/postprocess/core.py: "Array" expects no type arguments, but 2 given
• 🛑 foam/app/postprocess/core.py: Invalid type: try using Literal[1] instead?
• 🛑 foam/app/postprocess/core.py: Invalid type: try using Literal[2] instead?
• 🛑 foam/app/postprocess/core.py: Item "None" of "Optional[Foam]" has no attribute "fields"
• 🛑 foam/app/postprocess/core.py: "Array" expects no type arguments, but 2 given
• 🛑 foam/app/postprocess/core.py: Invalid type: try using Literal[1] instead?
• 🛑 foam/app/postprocess/core.py: Invalid type: try using Literal[2] instead?
• 🛑 foam/app/postprocess/core.py: "Array" expects no type arguments, but 2 given
• 🛑 foam/app/postprocess/core.py: Invalid type: try using Literal[0] instead?
• 🛑 foam/app/postprocess/core.py: Invalid type: try using Literal[1] instead?
• 🛑 foam/app/postprocess/core.py: Argument 1 to "probes" of "VTK" has incompatible type "Tuple[float, ...]"; expected "Tuple[float, float, float]"
• 🛑 foam/app/postprocess/core.py: Invalid index type "Tuple[float, ...]" for "Dict[Tuple[float, float, float], Dict[str, Array]]"; expected type "Tuple[float, float, float]"
• 🛑 foam/app/postprocess/core.py: "Array" expects no type arguments, but 2 given
• 🛑 foam/app/postprocess/core.py: Invalid type: try using Literal[0] instead?
• 🛑 foam/app/postprocess/core.py: Invalid type: try using Literal[1] instead?
• 🛑 foam/app/postprocess/core.py: Unsupported left operand type for - ("Array")
• 🛑 foam/app/postprocess/core.py: Value of type "Array" is not indexable
• 🛑 foam/app/postprocess/core.py: Incompatible return value type (got "Dict[Tuple[float, ...], Dict[str, Any]]", expected "Dict[Tuple[float, float, float], Dict[str, Array]]")
• 🛑 foam/app/postprocess/core.py: "Array" expects no type arguments, but 2 given
• 🛑 foam/app/postprocess/core.py: Invalid type: try using Literal[1] instead?
• 🛑 foam/app/postprocess/core.py: Invalid type: try using Literal[2] instead?
• 🛑 foam/app/information/core.py: Invalid index type "Type[Any]" for "Dict[ABCMeta, Callable[[Any], Any]]"; expected type "ABCMeta"
• 🛑 foam/app/command/adapter.py: Value of type "Optional[Data]" is not indexable
• 🛑 foam/app/command/adapter.py: Value of type "Optional[Data]" is not indexable
• 🛑 foam/app/command/adapter.py: Argument 1 to "findall" of "Pattern" has incompatible type "Optional[str]"; expected "str"
• 🛑 foam/app/command/core.py: Item "None" of "Optional[Path]" has no attribute "iterdir"
• 🛑 foam/app/command/core.py: Item "None" of "Optional[Path]" has no attribute "iterdir"
• 🛑 foam/app/command/core.py: Unsupported left operand type for / ("None")
• 🛑 foam/app/command/core.py: Argument 1 to "rmtree" has incompatible type "Optional[Path]"; expected "Union[str, bytes, PathLike[str], PathLike[bytes]]"
• 🛑 foam/app/command/core.py: Argument 1 to "save" of "Foam" has incompatible type "Optional[Path]"; expected "Union[str, Path]"
• 🛑 foam/app/command/core.py: Unsupported left operand type for / ("None")
• 🛑 foam/app/command/core.py: Item "None" of "Optional[IO[bytes]]" has no attribute "__iter__" (not iterable)
• 🛑 foam/init.py: Incompatible types in assignment (expression has type "Optional[str]", variable has type "str")
• ⚠️ foam/base/lib.py: Hint: "python3 -m pip install types-PyYAML"
• ⚠️ foam/base/lib.py: (or run "mypy --install-types" to install all missing stub packages)
• ⚠️ foam/base/core.py: Possible overload variants:
• ⚠️ foam/base/core.py: def get(self, <nothing>) -> None
• ⚠️ foam/base/core.py: def [_T] get(self, <nothing>, _T) -> _T
• ⚠️ foam/base/core.py: Left operand is of type "Optional[Path]"
• ⚠️ foam/app/postprocess/core.py: See https://mypy.readthedocs.io/en/stable/running_mypy.html#missing-imports
• ⚠️ foam/app/postprocess/core.py: Left operand is of type "Optional[Path]"
• ⚠️ foam/app/postprocess/core.py: Possible overload variants:
• ⚠️ foam/app/postprocess/core.py: def [_T] set(self) -> Set[_T]
• ⚠️ foam/app/postprocess/core.py: def [_T] set(self, Iterable[_T]) -> Set[_T]
• ⚠️ foam/app/postprocess/core.py: Possible overload variants:
• ⚠️ foam/app/postprocess/core.py: def sum(Iterable[bool], start: int = ...) -> int
• ⚠️ foam/app/postprocess/core.py: def [_SupportsSumNoDefaultT <: _SupportsSumWithNoDefaultGiven] sum(Iterable[_SupportsSumNoDefaultT]) -> Union[_SupportsSumNoDefaultT, Literal[0]]
• ⚠️ foam/app/postprocess/core.py: def [_AddableT1 <: SupportsAdd[Any, Any], _AddableT2 <: SupportsAdd[Any, Any]] sum(Iterable[_AddableT1], start: _AddableT2) -> Union[_AddableT1, _AddableT2]
• ⚠️ foam/app/command/core.py: Left operand is of type "Optional[Path]"
• ⚠️ foam/app/command/core.py: Left operand is of type "Optional[Path]"

Compare major version only

It is recommended to compare major version only, this allows you to change only the minor version each time you make a minor change that does not affect compatibility, making it easy to perform version control.

of.yaml/foam/core.py

Lines 41 to 44 in 4f71828

	if version < current:
	warnings.warn('Forward compatibility is not yet guaranteed')
	elif version > current:
	warnings.warn('Backward compatibility is not yet guaranteed')

临时中文说明书（Temporary Chinese manual）

https://docs.qq.com/doc/DUVRDYlVUeklIUmVl

operands could not be broadcast together with shapes (N,3) (N,)

of.yaml/foam/app/postprocess/core.py

Line 143 in 1137baf

field = self._cell_fields[key] * self._cell_fields['V']

Rename as pipeline

of.yaml/foam/command/core.py

Line 60 in 03db1fe

def pipe(self) -> t.List[str]:

`dictToFoam` executable binary file

Rust

[package]
name = "dictToFoam"
version = "0.1.0"
edition = "2021"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]
compress-tools = "0.12.2"
yaml-rust = "0.4"