Why does Django throw ImproperlyConfigured when I try to import in the Python shell?

This one kept throwing me when I started Django. Suppose you have a project project with an app app, and you want to inspect it. So you import parts of it in the Python shell:

(virtualenv) $ PYTHONPATH=. python
Python 3.9.2 (default, Feb 28 2021, 17:03:44) 
[GCC 10.2.1 20210110] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import project.app.views
(...cut traceback...)
django.core.exceptions.ImproperlyConfigured: Requested setting INSTALLED_APPS,
but settings are not configured. You must either define the environment variable DJANGO_SETTINGS_MODULE
or call settings.configure() before accessing settings.

Fine. So you set DJANGO_SETTINGS_MODULE:

(virtualenv) $ PYTHONPATH=. DJANGO_SETTINGS_MODULE=project.project.settings python
Python 3.9.2 (default, Feb 28 2021, 17:03:44) 
[GCC 10.2.1 20210110] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import project.app.views
(...cut traceback...)
django.core.exceptions.AppRegistryNotReady: Apps aren't loaded yet.

with no explanation as to what to do next.