matryer / is Goto Github PK
View Code? Open in Web Editor NEWProfessional lightweight testing mini-framework for Go.
License: MIT License
Professional lightweight testing mini-framework for Go.
License: MIT License
Would you consider adding these functions to the API? I could see it being useful for numeric comparisons but also understand that you want to keep the API to a minimum.
Thanks.
Otherwise, URLs appear as comments
Hi Mat!
Big fan (of Go of course ๐), long time listener of GoTime and working with Go full time for 4 years next month!
So... I just tried this nice little assert library and the first thing I notice is that my editor thinks I shouldn't do this:
is := is.New(t)
It says: Variable 'is' collides with imported package name
(And I hate to disappoint my editor!)
What is your defence? (Trying to be funny ๐)
It seems like there hasn't been a release that uses the IS_NO_COLOR
environment variable mentioned in the README. Would it be possible to tag a new release?
P.S. I enjoy listening to you and friends on Go Time! ๐
It would be cool to have variable names in the output.
Example:
is.Equal(signedin, true) // must be signed in
could be rendered like this:
is_test.go:175: not true: signedin<false> == true // must be signed in
go test
without -nocolor
works:
sam@macbook:~/go/src/github.com/sbward/s (master)
$ go test ./model/
permissions_test.go:11: model.Permission(2) != int(1)
--- FAIL: TestPermissionBits (0.00s)
FAIL
FAIL github.com/sbward/s/model 0.009s
But go test
with -nocolor
gives an error:
sam@macbook:~/go/src/github.com/sbward/s (master)
$ go test ./model/ -nocolor
flag provided but not defined: -nocolor
...
Is there a specific reason why this project uses the restrictive GPL 3 license? I for one would be able to use it in a much more relaxed way if it used a more open license like MIT.
@matryer, would you consider switching?
When comparing std big.Int
's or big.Float
's there is an edge case for the value zero where is
reports the error 0 != 0
. The reason for this is the big.{Int,Float}
attribute neg bool
which can be both true
and false
for the value zero.
package main
import (
"math/big"
"testing"
"github.com/matryer/is"
)
func TestBig(t *testing.T) {
is := is.NewRelaxed(t)
big0 := new(big.Int).Add(big.NewInt(-1), big.NewInt(1))
is.Equal(new(big.Int), big0) // big.Int
is.Equal(new(big.Float), big.NewFloat(0)) // big.Float
}
This will result in:
is_test.go:13: 0 != 0 // big.Int
is_test.go:14: 0 != 0 // big.Float
--- FAIL: TestBig (0.00s)
FAIL
It's not really a bug since the structs are different after all. However, since this are std types it would be nice if they would be compared by their Cmp
function.
I started switching from google/go-cmp
to is
in pursuit of the latter's more pleasant and succinct syntax, but unfortunately have run into issues with how times are compared by is
.
In the code being tested, a time is created using time.Now()
, stored to a database, and then later retrieved. When comparing the two times (or the structs containing time values), is.Equal
reports that the times are different because one has a monotonic counter and the other does not. This differs from how the standard library time.Equal()
functions, which is recommend by the Go team instead of directly comparing time values.
For example, consider the following excerpt where time1
has a monotonic counter, and time2
is the same time without a monotonic counter:
time1 := time.Now()
time2 := time1.Round(0) // returns time1 stripped of any monotonic clock reading but otherwise unchanged (see https://pkg.go.dev/time#Time.Round)
fmt.Printf("time1.Equal(time2) = %v\n", time1.Equal(time2))
is.Equal(time1, time2)
which results in the following:
time1.Equal(time2) = true
test.go:25: 2023-03-10 19:18:46.3692787 -0500 EST m=+0.083328801 != 2023-03-10 19:18:46.3692787 -0500 EST
Is there a better way to handle these comparisons in is
? Some options I can imagine include:
time.Equal()
if the values being compared are times.google/go-cmp
where all "Types that have an Equal method may use that method to determine equality".google/go-cmp
library itself to perform underlying equality comparisons.we're only getting a B because of bad gofmting: https://goreportcard.com/report/github.com/matryer/is#gofmt
The test panics with panic: runtime error: comparing uncomparable type map[string]string
if two nil slices or two nil maps are compared.
This can be simulated by adding the following test cases:
{
N: "Equal(nilSlice1, nilSlice2)",
F: func(is *I) {
var s1 []string
var s2 []string
is.Equal(s1, s2) // nil slices
},
Fail: "",
},
{
N: "Equal(nilMap1, nilMap2)",
F: func(is *I) {
var m1 map[string]string
var m2 map[string]string
is.Equal(m1, m2) // nil maps
},
Fail: "",
},
While I really like the small and simple API that is
has, one thing that makes it difficult to use is when developing utilities using is. Let me demonstrate:
func openDbHelper(is *is.I, dsn string) *sql.DB {
db, err := sql.Open("mysql", dsn)
is.NoErr(err) // error while opening database connection
return db
}
func CreateTestDatabase(is *is.I, dsn string) {
db := openDbHelper(is, dsn)
is.NoErr(db.Ping())
// ...
}
Let's say I use openDbHelper at different test functions. If openDbHelper
fails, I would only see this test output:
test_utils.go:222: failed to open database (or something)
This would only show me where the error occurs, but without more context, I end up never knowing exactly how the error happened, because I only have the file - line position of where the error happened, but I don't know who called that test utility.
I'd really enjoy having something like this instead:
// at testutils.go
func openDbHelper(is *is.I, dsn string) *sql.DB {
db, err := sql.Open("mysql", dsn)
is.NoErr(err)
return db
}
func CreateTestDatabase(is *is.I, dsn string) {
db := openDbHelper(is.WithFuncCall(), dsn)
is.NoErr(db.Ping())
// ...
}
// at somepackage_test.go
func TestSomething(t *testing.T) {
is := is.New(t)
CreateTestDatabase(is.WithFuncCall(), "....")
}
And, in this hypothetical case, if TestSomething
failed, you would get this more helpful error output:
test_utils.go:222: failed to open database (or something)
test_utils.go:(other line): testutils.CreateTestDatabase
somepackage_test.go:(another line): somepackage.TestSomething
This way I immediately know where the error happened, and I don't have to juggle around with go test -run ...
to check exactly where the error happened.
What do you think?
I sometimes have cases, where I would like to alter or extend the way is.Equal
is deciding, if the values are considered equal. After trying several approaches, I came up with the following solution, which does not extend the current API surface, but adds a maximum of flexibility and freedom to the user in regards to how is.Equal
decides if two inputs are equal.
The proposal is to value a newly defined matcher
interface, which is defined as follows (the matcher
interface does not need to become part of the public API of github.com/matryer/is
, in fact it can just be defined inline where needed):
type matcher interface{
Match(interface{}) bool
}
The actual change, that I am proposing is, to extend the existing areEqual
function like this (lines 9-11):
// areEqual gets whether a equals b or not.
func areEqual(a, b interface{}) bool {
if isNil(a) && isNil(b) {
return true
}
if isNil(a) || isNil(b) {
return false
}
if matcher, ok := a.(interface{ Match(interface{}) bool }); ok {
return matcher.Match(b)
}
if reflect.DeepEqual(a, b) {
return true
}
aValue := reflect.ValueOf(a)
bValue := reflect.ValueOf(b)
return aValue == bValue
}
In theory, this change does alter the working of is.Equal
. In practice, I assume the chances of this change having negative side effects for users of this package to be extremely low. Only users, that use this package to compare types, that implement this exact interface would be affected. I rate the profit of this change to be way higher than the risk of negative side effects.
If you think, that the current signature (Match(interface{}) bool
) is too likely to cause problems, the name of the method can easily be altered such that the chance of a collision become negligible (e.g. MatRyerIsMatch(interface{}) bool
๐).
If I find acceptance for this proposal, I am happy to provide the necessary PR to update the code, the tests and the documentation.
As a side note, the areEqual
function could be minimally simplified by replacing:
if isNil(a) && isNil(b) {
return true
}
if isNil(a) || isNil(b) {
return false
}
with
if isNil(a) || isNil(b) {
return isNil(a) && isNil(b)
}
The following test produces an incorrect message when failing, as the lines passed to decorate()
are not being escaped before being passed into log()
's Sprintf()
.
func TestFormatStringEscape(t *testing.T) {
is := New(t)
is.Equal("20% VAT", "0.2 VAT")
}
=== RUN TestFormatStringEscape
is_test.go:296: 20%!V(MISSING)AT != 0.2 VAT
--- FAIL: TestFormatStringEscape (0.00s)
Hey Mat, long time. Ran into a curious issue worth filing.
Running go test -json ...
and using this package reports incorrect test names in JSON events. This could very well be a bug in Go (test2json), but figured I'd start here because if I remove is
and use regular std. lib primitives it reports the correct thing.
I noticed this happening in https://github.com/pressly/goose and put together a PR to showcase the problem.
In this commit I purposefully changed a test to trigger a failure. Note the name of the test is TestNotAllowMissing
.
pressly/goose@9b7c732#diff-080f0d7ba408eaf6181e1f61388995ebcd57fdd1468ffafffbf789176531418fR30
When I run go test -v -json -count=1 ./tests/e2e -dialect=postgres
I happily get JSON output. But, the test name in the JSON event doesn't line up with the Output.
I was expecting Output that is
prints to be associated with test named TestNotAllowMissing
, but instead it was associated with another test named TestMigrateFull
. (the name is non-deterministic, it changes between runs).
The raw output can be found here . But I've copied and trimmed the relevant bits:
{"Test":"TestMigrateFull","Output":"\t\u001b[90mallow_missing_test.go:30: \u001b[39m7 != 8\n"}
{"Test":"TestNotAllowMissing","Output":"--- FAIL: TestNotAllowMissing (3.73s)\n"}
The first JSON event refers to allow_missing_test.go:30
, in the code base the call site on line 30 is located within the test named: TestNotAllowMissing
, but the test name is TestMigrateFull
.
The second JSON event correctly outputs the failure under the test named TestNotAllowMissing
.
This could very well be a bug in how go test prints JSON events, but I suspect it may be something within this library. I say this because if I remove is
from that specific test (TestNotAllowMissing
) I see the expected output associated with the correct Test name:
Raw output can be found here
{"Test":"TestNotAllowMissing","Output":" allow_missing_test.go:35: got 7: want: 8\n"}
{"Test":"TestNotAllowMissing","Output":"--- FAIL: TestNotAllowMissing (2.81s)\n"}
Hey there, thanks for creating this package. I use it in all my tests. Would you be open to removing the tab in front?
The misalignment with other printed logs bugs me a bit (very subjective of course):
And with the colors, I think there's enough visual separation:
If you're up for this change, I'd be happy to open a PR.
The current v1.0.0
release still contains the old GPL license. Could we create a new release with the new MIT license or move the existing release tag to include it?
Adding New
and NewRelaxed
methods to the I
object will allow subtests to work using the normal pattern:
func Test(t *testing.T) {
is := is.New(t)
t.Run("sub1", func(t *testing.T){
is := is.New(t)
is.Equal(1, 1) // should be equal
})
}
I would like to use the library, but it has no tag/release. Would be great to have semver tag (for vgo/dep, etc).
Thanks!
With the commit 46663c1 the parsing of the flags has changed to the use of flag.Parse()
.
Because flag.Parse()
is called in init()
, it prevents the source package (the one, that uses github.com/matryer/is
) from defining and using it's own flags, because the calls to e.g. flag.Bool(...)
will be after the execution of flag.Parse()
and therefore these flags are not considered and the execution of go test
fails with flag provided but not defined: ...
.
This leaves us with two options:
go test
, then it would also be save to remove the call to flag.Parse()
in init()
, because go test
will execute flag.Parse()
I think it is quite common to got caught into pitfalls getting errors like "unit8(0) != int(0)" (by calling is.Equal with an untyped constant for example).
Now that Go has added generics support, it is easy to add type safety support to equal, like:
func (is *I) EqualSafe[T any](a, b T) {
is.Helper()
return is.Equal(a, b)
}
I think it is a good time to make a new release ๐
Hi, let me first say that the library is great, thank you for creating it!
I find myself repeating few checks for errors that could be made into assertation.
It would be my pleasure to contribute this upgrades if we agree on their form.
is.NoErr(err)
is great but in lots of situations I need to check the opposite, that error is present.
Using is.True(err != nil)
works but does not have the same approach to assertation.
I suggest adding is.Err(err)
that would assert err
is not nil
.
Usually, I add context to errors but I do not use custom error types as there is no behavior associated with this errors. When I have this situation, I would like to check that error contains a particular substring.
I suggest adding is.ErrContains(err, "part of error message")
that would assert if error contains given substring.
What do you think about this two additions?
There are new commits, but no new release or tag for 2 years ...
Hi!
Watched the talk you gave on The Art of Testing
and really liked it. That's why I'm trying out this package, and I really like its simplicity.
One thing though that I'm missing is a Contains
method (basically this).
As there is none so far, I guess there is some other way to do it?
For example, I have a function that returns [][]string
, and I wanna check if some []string
are in that slice, but do not wanna specify the order.
Thanks!
The following code
if isNil(a) || isNil(b) {
is.logf("%s != %s", is.valWithType(a), is.valWithType(b))
} else if reflect.ValueOf(a).Type() == reflect.ValueOf(b).Type() {
is.logf("%v != %v", a, b)
} else {
is.logf("%s != %s", is.valWithType(a), is.valWithType(b))
}
can be simplified to
if isNil(a) || isNil(b) || reflect.ValueOf(a).Type() != reflect.ValueOf(b).Type() {
is.logf("%s != %s", is.valWithType(a), is.valWithType(b))
} else {
is.logf("%v != %v", a, b)
}
vim can't handle terminal escape codes in compiler/test output, and vim-go does not strip them out. A failing test is rendered in the Quickfix window as:
^[[90mhtml_test.go|14| ^[[39merr: unexpected input^[[32m // should succeed^[[39m
And tries to open a file called ^[[90mhtml_test.go
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.