Change failing tests result from "FAIL" to "FAILED"

[throwtheswitch]

Change failing tests result from "FAIL" to "FAILED"

[mvandervoord]

I like this.

I'm a little hesitant about merging it, only because I don't know what automation people have already wrapped around this. Maybe we make it optional? We'd default to the new behavior (because I think it is preferable) but add a define to support keeping the old? Is that too silly of a thing to have an option for?

[nscooling]

I have some python based automation with the current output, just looking for FAIL to be the first 4 characters in the line, e.g.
pos = line.find('FAIL')
if pos == 0:
break
This works with both FAIL and FAILED as I can use readline, so I'm hoping most automation won't be impacted.
But with the SWO output it's just byte streams over the debug interface, which can be broken anywhere in the text, so finding FAIL as the first 4 characters is a bit more challenging; i.e. finding occurrences of '/n' and then working from there (as it's written in C not Python). Doable, but with "FAILED" I can simply use strstr.

[nscooling]

I my code I simply added a new define:
const char UnityStrFail[] = "FAIL";
const char UnityStrFailed[] = "FAILED";

and changed
void UnityPrintFail(void)
{
UnityPrint(UnityStrFailed);
}

This way, if "FAIL" rather than "FAILED" is needed it is then a case of simply changing the #define

[xor-gate]

As @mvandervoord already said some time ago, it is better to keep as-is because people probably rely on its output saying PASS or FAIL at the end summary.

[nscooling]

@xor-gate I'm not quite sure that's why he said "Maybe we make it optional? We'd default to the new behavior (because I think it is preferable) but add a define to support keeping the old?" but I've given up on this anyway as there appears to be no demand to do full CI using Unity.

[mvandervoord]

Niall: Yes, I was proposing something like that. Adding a define to change to output to the alternative FAILED instead of the standard FAIL output.

I think your assumption about having no demand for people using Unity in Continuous Integration systems is incorrect. From what I understand, many of us ARE using it in CI systems, myself included. (I couldn't really fathom a guess as to percentage... but based on discussions on the forums I know Unity is being used in many popular CI frameworks like TeamCity, Jenkins, CruiseControl, and Travis,)

Actually, this is why I'd prefer to not change the output format on a whim. It might be simpler for you to parse it in your application (and I appreciate that), but there are a lot of people depending on the current output format and I'm not willing to break compatibility for them without good reason. I'm all for making life easier for people like yourself, which is why I proposed adding an option to support a different output.

There are also a number of people using full-blown output formatters to make Unity's output look like other standards (nUnit, google Test, etc). If you're not loving the current format, you might want to consider hunting for one of those. Eventually I hope to pull them all into Ceedling.

Anyway, if you'd like to submit a pull request which optionally changes the format, I'd be happy to merge it. Otherwise I'm sure we'll be getting to this at some point. I DO find this suggestion to be a useful one, it's just not a high priority at the moment.

[jsalling]

Mark,
What do want the option macro named? How about UNITY_PRINT_FAILED_VS_FAIL_AT_END?
I'm just not sure on this name.

To clarify, did you want it off by default?
Something like:
#ifdef UNITY_FLAGNAME // past tense, off by default
UNITY_OUTPUT_CHAR('E'); UNITY_OUTPUT_CHAR('D');
#endif

If you give me assistance with a good name, I'll make the pull request. :) Cheers.

[mvandervoord]

:) Do you find that one of the biggest challenges in programming is deciding on the proper name for things? I do.

How do you feel about UNITY_DIFFERENTIATE_FINAL_FAIL?

Otherwise, the behavior was exactly what I was suggesting.

Thanks for your help!

Mark

[jsalling]

I do, too.
Thanks for the name. I feel good about that choice.
You're welcome for the help. I'm glad to contribute.