Proper Test Suite

[tamasfe]

Proper Test Suite

[moorereason]

https://github.com/BurntSushi/toml-test has a new maintainer. It's been updated to TOML v1.0.0.

There's a simple command line tool used for testing the suite. I don't know jack about Rust, but I would love to see taplo provide a binary to work with the toml-test cmd.

[tamasfe]

@jasikpark thank you a lot! I'll look into these, we have some tests based on those 0.4.0 tests (and some 0.5.0) and bug reports, but tests are never enough.

[tamasfe]

toml-test can be used with taplo-cli from the master branch (cargo install --git=https://github.com/tamasfe/taplo.git --features toml-test taplo-cli and then toml-test taplo -- toml-test).

Unfortunately it is not part of the CI yet, and the cli has to be specifically built with the feature enabled, but I will include it in the next release by default as it barely adds anything to the final binary.

The results after the first run are not great, not terrible:

toml-test [taplo toml-test]: using embedded tests: 240 passed, 73 failed

As far as I see the causes of the failed tests roughly fall into three categories:

• actual bugs
• the parser being too permissive
• wrong string escape sequences

The latter two are not all that relevant for IDE uses, but might be essential for people who use taplo to validate TOML files (e.g. in CI).

Additional test cases and reports are welcome, fixes are even more welcome!

[jasikpark]

https://github.com/bd82/toml-tools/tree/master/packages/prettier-plugin-toml might be useful - they use Burntsushi's TOML test suite I believe?

https://github.com/BurntSushi/toml-test

[jasikpark]

Oh lol that repo is in sync with TOML 0.4.0 nvm.

[jasikpark]

Probably not a bad idea for testing against 0.4.0 tho? IDK anything, I just really appreciate this project

[jasikpark]

https://github.com/iarna/toml-spec-tests - tests for TOML 0.5.0 😅

[jasikpark]

https://github.com/iarna/toml-spec-tests/tree/1.0.0-rc.1 aha!