Supported Only Filter not working?

[fhir-crucible]

Supported Only Filter not working?

[okeefm]

Do we filter nightly runs by supported tests only?

[jimsteel]

The servers I was referring to in the chat are https://projectcrucible.org/servers/582f8e486372757e2e000000 (FHIR 1.8.0) and https://projectcrucible.org/servers/57d30d486372756382090000 (FHIR 1.4.0)

[arscan]

Thanks for reporting this @jimsteel. The problem is that our automated tests that run twice weekly do not obey the 'supported only' filter. I believe this was intentional, in case the server did not report a proper conformance. But this causes bad behavior in servers like yours, because you are getting a much lower 'grade' than you should because your server only claims to support a subset of the spec. So unless @okeefm or @jawalonoski have an issue with me changing this, I'll go ahead and update the nightly runs so that they only run tests that are supported by each server.

It'll take at least a few days to get this up into production, and to fix your history so that the unsupported test runs are erased. I'll update this issue when this is complete.

Thanks!