Super scaffolding any model causes the OpenAPI test to start failing

[jagthedrummer]

Super scaffolding any model causes the OpenAPI test to start failing

[jagthedrummer]

@newstler, wondering if you have any suggestions on how we can handle this better.

[jagthedrummer]

After poking around at this for a while I found that I could get the test to pass by updating test/factories/weidgets.rb.

It started like this:

FactoryBot.define do
  factory :widget do
    association :team
    name { "MyString" }
  end
end

And once I did this, the test started passing:

FactoryBot.define do
  factory :widget do
    association :team
    name { "MyString" }
    factory :widget_example do
      id { 42 }
      team { FactoryBot.example(:team) }
    end
  end
end

I think we either need to update the scaffolding process to automatically add the _example factory OR we need to figure out some other way to prevent scaffolding for instantly causing test failures that don't provide any useful info about what's wrong.

[jagthedrummer]

An example factory like this also works:

    factory :widget_example do
      id { 42 }
      team_id { 42 }
    end

(Hard coding a team_id instead of using FactoryBot.example to create an example team.

[jagthedrummer]

After looking at this some more I think maybe our test is just overly fragile. The "failures" are actually just letting you know about warnings issued by redocly. I'm not sure we should fail the tests just due to warnings.

These are the lines causing the problem:

bullet_train/test/controllers/api/open_api_controller_test.rb

Lines 18 to 20 in e26545d

	failures = output.match(/You have (\d+) warnings/) || output.match("Failed to parse api definition")
	puts output if failures
	refute failures

I'm currently thinking that we shouldn't lump together warnings and failures from redocly and treat them all as failures. Instead I think that we should separate them and only treat redocly failures as failures. And then just output a message if there are warnings.

Maybe something like:

    failures = output.match("Failed to parse api definition")
    puts output if failures
    refute failures

    warnings = output.match(/You have (\d+) warnings/)
    if warnings
      puts "-----------------".yellow
      puts warnings.yellow
      puts "-----------------".yellow
    end

@newstler do you see any problems with doing something like this?

[newstler]

Hey, I’m sorry I haven’t seen that due to absence of notifications on GitHub some why.

You’re right saying that the solution would be to update the scaffolding process to automatically add the _example factory.

I wouldn't change the test itself though, as redocly's warnings are basically OpenAPI errors, meaning the documentation is noncompliant.