pytest==6.1.0: processing filter warnings is done before django.setup()

[charlesaracil-ulti]

Lauching pytest results in the following error:

pytest test_foo.py click to expand

Traceback (most recent call last):
  File "/home/presto/.virtualenvs/presto/bin/pytest", line 8, in 
    sys.exit(console_main())
  File "/home/presto/.virtualenvs/presto/lib/python3.8/site-packages/_pytest/config/__init__.py", line 187, in console_main
    code = main()
  File "/home/presto/.virtualenvs/presto/lib/python3.8/site-packages/_pytest/config/__init__.py", line 143, in main
    config = _prepareconfig(args, plugins)
  File "/home/presto/.virtualenvs/presto/lib/python3.8/site-packages/_pytest/config/__init__.py", line 318, in _prepareconfig
    config = pluginmanager.hook.pytest_cmdline_parse(
  File "/home/presto/.virtualenvs/presto/lib/python3.8/site-packages/pluggy/hooks.py", line 286, in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
  File "/home/presto/.virtualenvs/presto/lib/python3.8/site-packages/pluggy/manager.py", line 93, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "/home/presto/.virtualenvs/presto/lib/python3.8/site-packages/pluggy/manager.py", line 84, in 
    self._inner_hookexec = lambda hook, methods, kwargs: hook.multicall(
  File "/home/presto/.virtualenvs/presto/lib/python3.8/site-packages/pluggy/callers.py", line 203, in _multicall
    gen.send(outcome)
  File "/home/presto/.virtualenvs/presto/lib/python3.8/site-packages/_pytest/helpconfig.py", line 100, in pytest_cmdline_parse
    config = outcome.get_result()  # type: Config
  File "/home/presto/.virtualenvs/presto/lib/python3.8/site-packages/pluggy/callers.py", line 80, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/home/presto/.virtualenvs/presto/lib/python3.8/site-packages/pluggy/callers.py", line 187, in _multicall
    res = hook_impl.function(*args)
  File "/home/presto/.virtualenvs/presto/lib/python3.8/site-packages/_pytest/config/__init__.py", line 1003, in pytest_cmdline_parse
    self.parse(args)
  File "/home/presto/.virtualenvs/presto/lib/python3.8/site-packages/_pytest/config/__init__.py", line 1280, in parse
    self._preparse(args, addopts=addopts)
  File "/home/presto/.virtualenvs/presto/lib/python3.8/site-packages/_pytest/config/__init__.py", line 1186, in _preparse
    self.hook.pytest_load_initial_conftests(
  File "/home/presto/.virtualenvs/presto/lib/python3.8/site-packages/pluggy/hooks.py", line 286, in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
  File "/home/presto/.virtualenvs/presto/lib/python3.8/site-packages/pluggy/manager.py", line 93, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "/home/presto/.virtualenvs/presto/lib/python3.8/site-packages/pluggy/manager.py", line 84, in 
    self._inner_hookexec = lambda hook, methods, kwargs: hook.multicall(
  File "/home/presto/.virtualenvs/presto/lib/python3.8/site-packages/pluggy/callers.py", line 208, in _multicall
    return outcome.get_result()
  File "/home/presto/.virtualenvs/presto/lib/python3.8/site-packages/pluggy/callers.py", line 80, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/home/presto/.virtualenvs/presto/lib/python3.8/site-packages/pluggy/callers.py", line 182, in _multicall
    next(gen)  # first yield
  File "/home/presto/.virtualenvs/presto/lib/python3.8/site-packages/_pytest/warnings.py", line 136, in pytest_load_initial_conftests
    with catch_warnings_for_item(
  File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__
    return next(self.gen)
  File "/home/presto/.virtualenvs/presto/lib/python3.8/site-packages/_pytest/warnings.py", line 52, in catch_warnings_for_item
    apply_warning_filters(config_filters, cmdline_filters)
  File "/home/presto/.virtualenvs/presto/lib/python3.8/site-packages/_pytest/config/__init__.py", line 1602, in apply_warning_filters
    warnings.filterwarnings(*parse_warning_filter(arg, escape=False))
  File "/home/presto/.virtualenvs/presto/lib/python3.8/site-packages/_pytest/config/__init__.py", line 1576, in parse_warning_filter
    category = warnings._getcategory(
  File "/usr/lib/python3.8/warnings.py", line 260, in _getcategory
    m = __import__(module, None, None, [klass])
  File "/home/presto/Projects/sandbox/pytest_bug_warning/bunny/models.py", line 10, in 
    class Bunny(models.Model):
  File "/home/presto/.virtualenvs/presto/lib/python3.8/site-packages/django/db/models/base.py", line 108, in __new__
    app_config = apps.get_containing_app_config(module)
  File "/home/presto/.virtualenvs/presto/lib/python3.8/site-packages/django/apps/registry.py", line 253, in get_containing_app_config
    self.check_apps_ready()
  File "/home/presto/.virtualenvs/presto/lib/python3.8/site-packages/django/apps/registry.py", line 136, in check_apps_ready
    raise AppRegistryNotReady("Apps aren't loaded yet.")
django.core.exceptions.AppRegistryNotReady: Apps aren't loaded yet.

The project is straighforward.

We created the following Warning and Model:

from django.db import models

# Create your models here.


class CarrotWarning(RuntimeWarning):
    pass


class Bunny(models.Model):
    pass

And use the following setup.cfg:

[tool:pytest]
filterwarnings =
    ignore::bunny.models.CarrotWarning

The bug was introduced by pytest==6.1.0, it worked fine with pytest==6.0.2.

We get that it seems to be introduced by pytest itself, but as its django-related, we assume that the fix will be in pytest-django.

We would guess from the changelog that it's linked to this issue : pytest-dev/pytest#6681

[bluetech]

Right, pytest now handles the warnings earlier, which means django doesn't get to init yet. But we didn't except warning types to rely on that.

IMO you should consider resolving this on your side -- instead of defining CarrotWarning in models.py, I'd move it to a standalone warnings.py module (or any other module which doesn't depend on django being initialized) and point the ignore to that. This way django doesn't need to be initialized. If the warning still needs to be available from models.py, you can just reexport it from there.

[charlesaracil-ulti]

Yes you're right I am able to dodge the bug by doing so :)

However I think they're might still be ground for a fix there, as the exception is highly unexpected and difficult to traceback. Could be a way to introduce django.setup() preventively so that the exception can't happen.

[bluetech]

I think we can probably improve the error message, but I don't think we can fix it -- the idea of the pytest change was exactly to set up the warnings before everything else.

I'll close this issue and open a pytest issue to track improving the error.

[bluetech]

pytest-dev/pytest#7864

[charlesaracil-ulti]

Okay thanks for the head's up ;)

[ewjoachim]

I could be wrong but I believe that even if the pytest bug was to be solved (and the import would not outright crash pytest), we would still end up with the warning filters not installed. I believe that in order for this to work, we need to somehow call django.setup() earlier, and that might be something to solve in pytest-django...

Meanwhile, I haven't found a practical example where moving the warning definitions higher in the package, in a module without import-time side effects didn't do the trick, so it's also perfectly fine to keep this as the official solution (I fear the day where the problematic warning import will be in a 3rd party, though).

Should we document this in pytest-django ? Or somehow try and solve this ?