About comparison testing

[christopheradams]

About comparison testing

[christopheradams]

This may be too obvious, but something like:

Testing

• When writing ExUnit assertions, be consistent about the order of the expected and actual values under testing. Prefer placing the expected result on the right, unless the assertion is a pattern match.

  # preferred - expected result on the right
  assert actual_function(1) == true
  assert actual_function(2) == false
  
  # not preferred - inconsistent order
  assert actual_function(1) == true
  assert false == actual_function(2)
  
  # required - the assertion is a pattern match
  assert {:ok, expected} = actual_function(3)
  

[ericmj]

The great thing about the assert macro is that it doesn't enforce the actual, expected positions and that was actually one of the design decisions behind it because sometimes it's hard to determine which expression is "expected" and which is "actual".

It feels backwards to try to enforce a rule in a style guide that goes against the design decision of the tool.

[christopheradams]

Can you link to any examples of both styles?

Even though the assert macro doesn't care, obviously, there are a few reasons I would tend to favor assert expected == actual:

• tradition (other testing libraries that take args and use that order)
• if you use a pattern match instead of a truthy value, the expected pattern is on the left
• assertion errors are often written/understood in the form "expected A but got B"

[marocchino]

sure.

• actual == expected
• expected == actual

[christopheradams]

From the looks of it, every test module is internally consistent (they pick one order and stick with it), but there are examples of both styles.

I'd guess the motivation behind putting actual first is to highlight "this is what is being tested". For example StringTest.

Would love to hear some more opinions.

[christopheradams]

The great thing about the assert macro is that it doesn't enforce the actual, expected positions

Sure, but the style guide is about enforcing things the compiler or runtime don't care about. :)

I tend to think that expected first has better meaning but actual first has better style (especially when successive lines test the same function over and over). Therefore, it seems to be a wash. That would recommend against adding a rule for this.

[eksperimental]

lexmag's elixir style guide is about to include this rule
lexmag/elixir-style-guide#11

[christopheradams]

Thanks for the pointer @eksperimental

For context the linked issue recommends that the "expected result goes on the right", in other words: actual == expected

Since this guide doesn't try to be unnecessarily "opinionated", I'm not sure if there's a way forward here.

[DavidAntaramian]

@christopheradams I don't think the style guide needs to be opinionated about it, just point out "be consistent."

As pointed out in the linked issue, one of the problems with the phrasing is that it will always be expected = actual when performing a pattern match as a test acceptance.

[christopheradams]

Added a new testing section with this actual == expected guideline. Thanks everyone!