ozontech/cute

Library for simply creating HTTP tests in Go with Allure reports.

cute's Issues

Update allure-go v0.6.18

Update to

github.com/ozontech/allure-go/pkg/allure v0.6.4
github.com/ozontech/allure-go/pkg/framework v0.6.18
github.com/ozontech/allure-go v0.6.18

json.Present() return error in case of 0

Often have need to use this assert, but 0 could be proper value too, so please, fix this.

retries for failed tests

please add retries for failed tests within one suite. It would be nice to add something like t.retry tags or something like that

lets lint only diffs with master branch

--new-from-rev=origin/master

Init README_ru.md file with Russian description of library usage

create module for telegram chat report

generate a report that will provide a link to the gait report, the number of tests passed, broken tests and bad tests

Current version CUTE (v0.1.0) does not work with github.com/ozontech/[email protected]

Test launch crushed:
../../../go/pkg/mod/github.com/ozontech/[email protected]/test.go:114:22: too many arguments in call to common.NewT
have (*testing.T, string, string)
want (provider.TestingT)

Can't read json_path syntax in AssertBody

code:
OptionalAssertBody( json.Equal("$.entries[?(@.role == 'HR-партнер')].event", "Анкета согласована"), ).
error:
test.go:178: [OPTIONAL ERROR] evaluating '$.entries[?(@.role == 'Сторонняя система проверки СБ')].event' resulted in error: 'parsing error: $.entries[?(@.role == 'Сторонняя система проверки СБ')].event :1:23 - 1:54 could not parse string: invalid syntax'

This syntax is corresponds to the library JsonPath, but using this in 0.1.8 make error

go: github.com/ohler55/[email protected]: invalid version: unknown revision v1.12.9

The module requests a dependency that apparently no longer exists, because the last tag that is in the specified repository is 1.15.0 (but it is available at https://pkg.go.dev/github.com/ohler55/[email protected])

.../go list -modfile=.../go.mod -m -json -mod=mod all #gosetup
go: github.com/ohler55/[email protected]: invalid version: unknown revision v1.12.9

It only helps to forcibly specify a newer version 1.21.1

require (
    ...
    github.com/ozontech/cute v0.1.14
    ...
)

require (
    ...
    github.com/ohler55/ojg v1.21.0 // indirect
    ...
)

add log management

Logging becomes uninformative when using paralleltest. We can solve the problem by using short logs or by disabling them.
Example: step: <name_step>(<status_code>)
Perhaps a mini log constructor would solve the problem?

Possibility to not fail the test during retries

cute/roundtripper.go

Lines 103 to 108 in 9389b60

	return cuteErrors.NewAssertError(
	"Assert response code",
	fmt.Sprintf("Response code expect %v, but was %v", it.Expect.Code, resp.StatusCode),
	resp.StatusCode,
	it.Expect.Code)
	}

Would it be possible to have a way to configure this part to return optional error.

For example, here:

cute/test.go

Lines 56 to 58 in 9389b60

	Count int
	Delay time.Duration
	}

type RequestRepeatPolitic struct {
        Count int
        OptionalError bool
        Delay time.Duration
}

Add an OptionalError bool that, if defined to true, will return cuteErrors.NewOptionalError instead of cuteErrors.NewAssertError
If the count has reached its last index, then switch back cuteErrors.NewAssertError and ultimately fail the test

problem with ExpectJSONSchemaFile

reproduction steps:

1. create json local file, example:
   { "name" : "Elton", "adult" : true }
2. use ExpectJSONSchemaFile in my test
3. in response has json like this:
   { "name" : "Ivan", "adult" : false }

expected result:
my test will have a fallen status, because name ≠ Elton and adult ≠ true

actual result:
my test is passed

If ExpectJSONSchemaFile only validate response by json schema, but not checking value, please add something like this for check json character by character

fix link in README on line "See full example here"

now there is 404 NOT FOUND

add error parsing to cute

Full error information is required to speed up the process of debugging tests.

Не прокидывается httpClient и jsonMarshaler для табличный тестов

Проблема возникает при создании тестов по инструкции: https://github.com/ozontech/cute?tab=readme-ov-file#table-tests
Создаётся []*cute.Test, с приватными полями httpClient и jsonMarshaler, которые дальше по коду нигде не прокидываются.

При вызове

cute/test.go

Line 434 in 9c6558e

body, err = it.jsonMarshaler.Marshal(o.bodyMarshal)

появляется panic: runtime error: invalid memory address or nil pointer dereference, что немного не приятно...

Misleading documentation in the README

Readme file contains misleading information about Suite cases.

Provided example does not work:

func (i *ExampleSuite) BeforeAll(t provider.T) {
    // Prepare http test builder
    i.testMaker = cute.NewHTTPTestMakerSuite()

    // Preparing host
    host, err := url.Parse("https://jsonplaceholder.typicode.com/")
    if err != nil {
        t.Fatalf("could not parse url, error %v", err)
    }

    i.host = host
}

Package cute contains no NewHTTPTestMakerSuite, instead it has NewHTTPTestMaker.

Please update current documentation on suites

Fix cuteErrors.NewAssertError function

Прошу исправить функцию cuteErrors.NewAssertError, сейчас если в ней указывать параметры actual interface{}, expected interface{}, то они не выводятся в ответе (и в результе выполнения теста)

Empty response body in allure reports

Not correct allure report in multi step tests

If use multi step feature in Suite:

func (i *ExampleSuite) TestExample_TwoSteps(t provider.T) {
	var (
		testBuilder = i.testMaker.NewTestBuilder()
	)

	testBuilder.
		Title("TestExample_TwoSteps").
		Tags("TestExample_TwoSteps", "some_tag").
		Parallel().
		CreateWithStep().

		// CreateWithStep first step

		StepName("Creat entry /posts/1").
		Request(req).
		ExpectExecuteTimeout(10*time.Second).
		ExpectStatus(http.StatusCreated).
		AssertBody(
			// Custom assert body
			examples.CustomAssertBody(),
		).
		ExecuteTest(context.Background(), t).

		// CreateWithStep second step for delete

		NextTestWithStep().
		StepName("Delete entry").
		RequestBuilder(
			cute.WithURL(u),
			cute.WithMethod(http.MethodDelete),
			cute.WithHeaders(map[string][]string{
				"some_auth_token": []string{fmt.Sprint(11111)},
			}),
		).
		ExecuteTest(context.Background(), t)
}

Result:

But if create test is't in suite

func TestExampleSingleTes_Witouttitle(t *testing.T) {
	cute.NewTestBuilder().
		Title("TestExample123123123132112313").
		Tags("TestExample_TwoSteps", "some_tag").
		CreateWithStep().
		StepName("Create entry"). // Создание 1 шага
		RequestBuilder(
			cute.WithURI("https://jsonplaceholder.typicode.com/posts/1/comments"),
			cute.WithMethod(http.MethodPost),
		).
		ExpectStatus(http.StatusMultiStatus).
		AssertBody(
			// Custom assert body
			CustomAssertBody(),
		).
		ExecuteTest(context.Background(), t).
		NextTestWithStep().
		StepName("Delete entry"). // Создание 2 шага
		RequestBuilder(
			cute.WithURI("https://jsonplaceholder.typicode.com/posts/1/comments"),
			cute.WithMethod(http.MethodDelete),
		).
		ExecuteTest(context.Background(), t)
}

First step will not create in allure report.

json asserts are not working with array values

json.Contains with jsonPath with such a query like $.items[*].title searches only the element with index 0, and not the entire list of elements

Log information about request after make request

Add retry and subtest parallelism capability

Hi @siller174

I initially thought of forking this but I thought it may benefits others, so I'm raising a PR for this one.
Basically, it's linked to #70
I added retry capability for tests and parallel capability for subtests to solve what I mentioned in the comments of the above issue 70.

Here is the PR:
#81

Open to comments and discussions of course, thanks for your time!

New release

So.... Time to change something.

1. Update linter
2. Update allure-go
3. Support new allure labels

func (it *cute) Titlef(format string, args ...interface{}) AllureBuilder 
func (it *cute) Descriptionf(format string, args ...interface{}) AllureBuilder 
func (it *cute) Stage(stage string) AllureBuilder 
func (it *cute) Stagef(format string, args ...interface{}) AllureBuilder 
func (it *cute) Layer(value string) AllureBuilder 
func (it *cute) TmsLink(tmsLink string) AllureBuilder 
func (it *cute) TmsLinks(tmsLinks ...string) AllureBuilder

4. Support provider.StepCtx

func TestInsideStep(t *testing.T) {
	runner.Run(t, "Single test with allure-go Runner", func(t provider.T) {

		t.WithNewStep("First step", func(sCtx provider.StepCtx) {
			sCtx.NewStep("Inside first step")
		})

		t.WithNewStep("Step name", func(sCtx provider.StepCtx) {
			u, _ := url.Parse("https://jsonplaceholder.typicode.com/posts/1/comments")

			cute.NewTestBuilder().
				Title("Super simple test").
				Tags("simple", "suite", "some_local_tag", "json").
				Parallel().
				Create().
				RequestBuilder(
					cute.WithHeaders(map[string][]string{
						"some_header": []string{"something"},
					}),
					cute.WithURL(u),
					cute.WithMethod(http.MethodPost),
				).
				ExpectExecuteTimeout(10*time.Second).
				ExpectStatus(http.StatusCreated).
				ExecuteTest(context.Background(), sCtx) // <---- Execute test with provider.StepCtx
		})
	})

}

5. Remove EnableHardValidation
6. Fix bug with custom http.Client
7. Implement new library for jsonPath

The linter 'golint' is deprecated (since v1.41.0) due to: The repository of the linter has been archived by the owner. Replaced by revive.

RequestRepeat forms curl without body

When we use RequestRepeat , then all requests except the first one are formed without a body

require and assert

Usually in test framework for hard and soft checks use Require and Assert word. Now you can turn on all checks to hard or soft mode.
Please, fix it

Update libs

Fix cuteErrors.NewAssertError function

Прошу исправить функцию cuteErrors.NewAssertError

сейчас если передавать параметры actual interface{} и expected interface{}, то в ответе они не возвращаются (и в результате теста тоже)

пример:

Добавить жесткий assert для статус кода

Тест продолжает исполняться даже если ExpectStatus(http.StatusOK) падает.
Ожидаешь 200, а получаешь 500, но тест шагает дальше.
Хотелось бы иметь жесткую проверку.

readme update

Ability to hide secrets in the allure

Now allure reports get the secrets in the open form from the headers and bodies.
Please add ability to hide secrets in the allure

Finalization of the project on multipart/form-data

Please add the project to multipart/form-data (add to query builder)

Getting an error message "Response is nill" when a request times out (cute v 0.1.20)

After updating to the cute v0.1.20 getting an error message in Allure [HTTP] Response is nil, when a request times out.
There is no detailed description of the error. It is difficult to determine the cause of the error from the description in Allure and Goland.

Are there errors that make a type test broken?

Suggestion to correct mistakes in "about" section

Hey, fellow ozon developers 👋
I suggest, you may update the text of "about" section of this repository and correct mistakes in the use of English:

Library for simply creates HTTP tests in Go with Allure reports. ->
Library for simply creating HTTP tests in Go with Allure reports.

It would be much better English 🙂
Cheers ✌️

[v0.1.1] ValidateJSONSchema returns only 1 error, but should return several errors.

ValidateJSONSchema returns only 1 error, but should return several errors.

	body := []byte(`
	{
		"firstName": "Boris",
		"lastName": "Britva",
		"age": "1"
	}
	`)

       jsSchemaString = `
	{
	  "$id": "https://example.com/person.schema.json",
	  "$schema": "https://json-schema.org/draft/2020-12/schema",
	  "title": "Person",
	  "type": "object",
	  "properties": {
	    "firstName": {
	      "type": "string"
	    },
	    "lastName": {
	      "type": "integer"
	    },
	    "age": {
	      "type": "integer"
	    }
	  }
	}

[Refactoring] Refactor Expect methods

Reimplement these methods like Require and Assert, e.g. ExpectJSONSchemaString => AssertJSONSchemaString and RequireJSONSchemaString

func (qt *cute) ExpectExecuteTimeout(t time.Duration) ExpectHTTPBuilder {
	qt.tests[qt.countTests].Expect.ExecuteTime = t

	return qt
}

func (qt *cute) ExpectStatus(code int) ExpectHTTPBuilder {
	qt.tests[qt.countTests].Expect.Code = code

	return qt
}

func (qt *cute) ExpectJSONSchemaString(schema string) ExpectHTTPBuilder {
	qt.tests[qt.countTests].Expect.JSONSchema.String = schema

	return qt
}

func (qt *cute) ExpectJSONSchemaByte(schema []byte) ExpectHTTPBuilder {
	qt.tests[qt.countTests].Expect.JSONSchema.Byte = schema

	return qt
}

func (qt *cute) ExpectJSONSchemaFile(filePath string) ExpectHTTPBuilder {
	qt.tests[qt.countTests].Expect.JSONSchema.File = filePath

	return qt
}

Logs in console

We need to be able to output logs to the console when running locally, or instructions on how to add them

[HOTFIX] "AfterExecute" not working after first test

Fix #26

Please, think about additional assert types

Currently we have next asserts:

• Contains
• Equal
• NotEqual
• Length
• GreaterThan
• LessThan
• Present
• NotPresent

Please, think to add next assert types:

• GreaterEqual
• LessEqual
• AssertArray

Add assert for json respone

Response fields data usually differ for different requests, but it would be nice to have ability to check full body data with one check. It will also help to avoid huge amount of asserts in test case.

Please add launch of individual test cases by tags

Now we are running all tests by suite, I would like to be able to add individual tags to those tests that we want to run, regardless of which suite it belongs to.

Could not forward custom HTTP client

add error types

Error typing is necessary to separate possible status code errors from others.

TraceID output to log after test execution

Please add traceID output to log after test execution

Proper timeout logging and handling

Currently, when request is timing out because of deadline exceeded, the logging is not very clear:

With the PR #79 , here is the result: