Legacy Crucible Tests prevent 100% conformance and success

[fhir-crucible]

Legacy Crucible Tests prevent 100% conformance and success

[jawalonoski]

Thanks for the issue report... we've been thinking about this... it also relates to systems having 100% when they've only ran a few tests, where if they ran all the tests, they might end up at 23% (for example). One thought we had was to move away from a "percentage" to a "score" so we could weigh all these factors.

Whatever the solution, we do agree we need to do something about this.... more soon.

[lawley]

When you're reviewing this, please take into account specialised system such as a specific terminology-only server that would only ever claim to implement a small number of resource types and associated operations.

Perhaps it would be worth defining several common profiles and reporting scores relative to these profiles. Thus you could say something like system X implements x% of the Terminology Services profile and passes y% of the tests that match that profile.

[jawalonoski]

Well, today you can see whether or not a server passes all the terminology related portions of the spec, but it does take some navigating. We have also considered moving towards a "badge" system... have a badge for terminology services, another for provider directory, etc. Not there yet....