Ship tests/devtools as extras

[pycalphad]

Ship tests/devtools as extras

[richardotis]

I also looked into if it would be possible to get rid of pycalphad/tests/datasets.py and just ship the original TDB files directly as pkg_resources, but it actually looks pretty nasty to get access to the TDBs at run-time and keep .egg compatibility (since you can't always assume your module files are separate files on disk):
http://peak.telecommunity.com/DevCenter/PythonEggs#accessing-package-resources
Accessing package data via ResourceManager API (for setuptools): http://peak.telecommunity.com/DevCenter/PkgResources#resourcemanager-api

[bocklund]

As an alternative to shipping the tests, we can easily let pycalphad developers get the required packages for testing and benchmarking with a requirements.txt file. That way the testing and benchmarking can not be installed by users through PyPI/conda-forge.

We discussed (and this issue until now has considered) shipping tests to be able to validate user installations if there are any issues, but I think this will become less important as pycalphad continues to become more stable.

An example requirements.txt file would contain just asv, nose, coverage, ..., while the dependencies for pycalphad (numpy, dask, ...) should continue to live in setup.py

[bocklund]

If we really wanted to do this so people who did not install in develop mode can run the test suite, I don't think that's relevant. The test suite should always pass for releases on all systems.

[richardotis]

Given that we're taking benchmarking in a different direction and we already do automated testing on most configurations, I do not think it's worth complicating our build/release process to make this work as an optional add-on.

[bocklund]

Closing this and putting a task to drop asv on the roadmap.

[amjokisaari]

regarding adding tests, my motivation for this is to verify my python environment / PATH to needed tools (e.g., as found with my running the examples...).