Prevent spurious breakage when tagging happens

[Geod24]

Prevent spurious breakage when tagging happens

[MartinNowak]

Projects that regularly regularly release broken test-suites should be removed from the project-tester.
If OTOH projects regularly get broken by dmd/druntime/phobos changes, then those failures are legitimate and should rarely be worked around.

[Geod24]

Projects that regularly regularly release broken test-suites should be removed from the project-tester.

The thing is, it's not broken for them. They do a legitimate change to their project test suite. I've never seen a project release a broken release, it was always some change in the settings (e.g. installing a new dependency in their .travis.yml) that broke stuff.

[MartinNowak]

So you're talking about that angle. Indeed new dependencies might break builds, do you have a good example? I think the simplest way to solve this for now is by communication, there are not so many projects with external dependencies (as those hardly integrate with dub anyhow).

Also we've pre-installed quite a list of dependencies and I just linked to that in the README (https://github.com/dlang/ci/blob/master/README.md#pre-installed-dependencies-on-buildkite-agents).

[PetarKirov]

Why not simply require that all projects are build in containers? That way won't have any dependency management to do on our side, except for the container runtime (e.g. Docker, Windows Containers, etc.)

[Geod24]

@MartinNowak : The only breakages I've seen so far have been just this.
A dependency to the build gets added, but we don't have it.
E.g. we had meson builds being added in 2 cases ( #307 (comment) ), and now a rewritten test-suite.