Custom Runner e TestCase

[rodrigondec]

Custom Runner e TestCase

[kingbuzzman]

I wrote a VERY basic PR to get the ball rolling. It should work.

@bittner any idea when behave 1.2.7 will be released? The issue is I'm having a hell of a hard time testing this through tox, since it validates it against pypi, and v1.2.7.dev2 is not pushed there, it fails. This line works fine locally, but tox hates it.

Ps. @bittner what's going on with the behave-django CI? My tests are not running for some reason (not like they will pass due to the aforementioned comment above).

-- Yes, the PR needs a little love.

[kingbuzzman]

Using a custom runner in the behave conf files doesn't fully work

@kingbuzzman can you confirm this?

I can

because behave-django does monkey patching before behave is called

Not because of this though. It's becuase we're using django's TestRunner classes in order to setup the db... and whatever else django does internally AND we monkey patch the TestRunner class we receive from behave.

Why does it work for your use case?

Because I needed to modify the way behave was working, ie, the way it was loading where the feature files were/ step file location, and my use case excluded django entirely. Never needed to change anything in TestCase.

The greatest issue of all is that django's TestRunner is absolutely 100% incompatible with behave's TestRunner.

https://github.com/django/django/blob/stable/4.0.x/django/test/runner.py#L550 (django's "TestRunner")
https://github.com/behave/behave/blob/v1.2.7.dev2/behave/runner.py#L728 (behave's "TestRunner")

As you can clearly see.. there is very little in common

[rodrigondec]

About this

@rodrigondec I don't want to disappoint you. If we have your use case covered by a feature of Behave itself, though, it would be unwise to add code to this repository to support doing the same thing. What do you think?

@danickfort-liip nailed the motive pretty well.

@bittner Using a custom runner in the behave conf files doesn't fully work, because behave-django does monkey patching before behave is called

I'm not looking for extending or customizing the behave aspect. I want to load my super complex Django TestCase onto behave-django.

It's not about behave itself. It's about the django DB setup and so on...

[rodrigondec]

I think #130 fixes this issue, can this be closed now @rodrigondec ?

This indeed solves the issue about the Runner class. Consequently it should resolve the TestCase issue as @bittner mentioned in #123 (comment)

At this point I'm no longer using behave-django. We decided to use the manual integration from behave seeing as my case may be off the charts compared to the "common" way.

Therefore, for me, this issue can be closed.

@kingbuzzman ty for the PR 🙏

[bittner]

Thank you for the effort you put in this issue!

Can you please check whether #78 is a duplicate of your request?

Custom runners are now supported by behave itself, out-of-the-box. Please install behave from the master branch to take advantage of the new feature.

[rodrigondec]

#78 is almost a duplicate indeed.

The difference is in the objective, approach and consequences.

I could use the manual integration explained on behave docs and it would work for me.

But I would prefer to use behave_django to avoid more lines and more heavy-lifting

My mindset was to not change ANY interface or CLI on the lib. The goal is to offer a slightly different implementation which could be more extensible/customizable. Instead of a new arg on CLI or new feature.

For example, with a extra custom behave hook we could provide a point on the code to do a monkey patch (if the dev wanted).

Main difference when I say "custom runner"

behave_django uses its own 'django runner' for compatibility between Django and Behave.

When I say custom runner, I'm meaning it comparing it with Django. I don't want to use the Django default runner.

I don't want to overwrite the runner for behave django.

I don't want to pass a custom runner to the behave django CLI (management command).

SimpleTestRunner could simply inherit the Django Runner dynamically instead of using DiscoverRunner directly.

Thats a Django approach from it's own documentation.

90% (my guess) of the cases or users of Django don't overwrite the DiscoverRunner or use the TEST_RUNNER configuration. The others ovewrite it and configure.

My point on this is

why use a hardcoded Django Runner instead of getting it dynamically as Django itself does?

This way, it doesn't matter if a Django project is using DiscoverRunner or a MyCustomDjangoCompliedRunner! The lib automagically uses the configured RUNNER to create the respectively behave_django runner.

I need to use a custom TestCase as well

I use a CustomTestCase instead of Django TestCase.

For this to be extensible/configurable, a monkey patch entry point would suffice.

[bittner]

Note that you can configure a custom runner directly with Behave, this doesn't need to be a CLI option, you can use the runner_class option in a configuration file that Behave recognizes. This feature was added with PR behave/behave#795.

For everything else that you'd like to make more automatic, more dynamic - yes! - please go forward and try a PR. I'm a big fan of avoiding excessive setup code. 👍

Be sure to put everything that is not strictly related to Django (or very specific to integrating with it) possibly into the Behave project itself. Ideally we keep our integration layer as slim as possible.

[rodrigondec]

Update?

[bittner]

@rodrigondec I don't want to disappoint you. If we have your use case covered by a feature of Behave itself, though, it would be unwise to add code to this repository to support doing the same thing. What do you think?

Can you please verify hands-on whether you can use your custom runner by simply adding runner_class = MyCustomRunner to the behave configuration? See behave.readthedocs.io/en/latest/behave.html for related docs. Make sure you have Behave installed from the master branch (not from PyPI).

[danickfort]

@bittner Using a custom runner in the behave conf files doesn't fully work, because behave-django does monkey patching before behave is called

[bittner]

@bittner Using a custom runner in the behave conf files doesn't fully work, because behave-django does monkey patching before behave is called

@kingbuzzman can you confirm this? Why does it work for your use case?

[danickfort]

@kingbuzzman and @rodrigondec are 100% right, it's about having the ability to run a custom Django TestCase. In my
particular case, I have to overload Django's LiveTestServerCase's behavior, which can't be achieved with behave's TestRunners.

[rodrigondec]

@kingbuzzman and @rodrigondec are 100% right, it's about having the ability to run a custom Django TestCase. In my particular case, I have to overload Django's LiveTestServerCase's behavior, which can't be achieved with behave's TestRunners.

Thats exactly my situation too. Created my own DjangoTestCase and use on my whole system.

And just to inform, I opened the PR #123 about this too!

[kingbuzzman]

@bittner friendly ping ;)

[kingbuzzman]

@bittner poke.

[kingbuzzman]

@bittner ??

[bittner]

I'm willing to merge the simpler solution that solves this issue. Simple in the sense of both low footprint in the code and simple for the user / the developer.

[kingbuzzman]

I think #130 fixes this issue, can this be closed now @rodrigondec ?

[bittner]

Let's be sure that the setup we now have satisfies everyone's use case here. Can you check, @rodrigondec?

See the updated docs for usage instruction of the new feature. 📘 🔖