How Django keeps track of your model classes?

I have ran the python manage.py makemigrations command several hundreds of times. Recently I had a surge of curiousity to find out how does django detect changes and generate migration files for an app.

I needed many answers but one of the first question I had was "How does Django keep track of my model classes?"

In Django, the creation of model classes is fundamental to mapping your application's data model to a database. These model classes inherit from Django's models.Model base class, which provides them with essential database interaction capabilities.

Under the hood, Django employs metaclasses to achieve its model detection magic.

But what exactly is a metaclass in Python?

Simply put, a metaclass allows you to customize class creation in Python. In case of Django's model system, the metaclass responsible for this process is called ModelBase.

Within the ModelBase metaclass, a crucial method known as __new__ plays a central role. This method invokes the register_model function, which, in turn, ensures that a given model registered within the app.

# django/django/db/models/base.py
class ModelBase(type):
    """Metaclass for all models."""

    def __new__(cls, name, bases, attrs, **kwargs):
        super_new = super().__new__

        ...
        new_class._prepare()
        # this is where a model class gets registered.
        new_class._meta.apps.register_model(new_class._meta.app_label, new_class)
        return new_class

But when does this new method on ModelBase get triggered?

To my surprise I found out that it gets triggered at import time. When you import a model class, new method is fired and Django's model detection process initiates. For example:

from myapp.models import MyModel

Behind the scenes, Django's machinery begins to work its magic during this import.

So, where does Django import all these models?

The answer lies in django.setup(), the entry point for various Django commands such as shell, makemigrations, migrate, runserver, and more.

Within django.setup(), the apps.populate(settings.INSTALLED_APPS) function registers all of your INSTALLED_APPS.

# django/django/__init__.py
def setup(set_prefix=True):
    """
    Configure the settings (this happens as a side effect of accessing the
    first setting), configure logging and populate the app registry.
    Set the thread-local urlresolvers script prefix if `set_prefix` is True.
    """
    ...
    apps.populate(settings.INSTALLED_APPS)

When I followed the code to the populate method, I could see it is defined under the Apps class. Within this method there is a method call as app_config.import_models(). The app_config derives from the config class we define for every new django app under apps.py.

# django/django/apps/registry.py
class Apps:
    """
    A registry that stores the configuration of installed applications.

    It also keeps track of models, e.g. to provide reverse relations.
    """

    def __init__(self, installed_apps=()):
        ...
        ...
        if installed_apps is not None:
            self.populate(installed_apps)

    def populate(self, installed_apps=None):
        """
        Load application configurations and models.

        Import each application module and then each model module.
        """
        ...
        ...
            # Phase 2: import models modules.
                for app_config in self.app_configs.values():
                    app_config.import_models()

And finally to where the place where "magic" happens. On the AppConfig class we have a method import_models which reads the MODELS_MODULE_NAME which by default is set to models.py and imports the whole file as a module. Thereby importing all the model classes in it.

# django/django/apps/config.py
class AppConfig:
    """Class representing a Django application and its configuration."""
    def import_models(self):
        # Dictionary of models for this app, primarily maintained in the
        # 'all_models' attribute of the Apps this AppConfig is attached to.
        self.models = self.apps.all_models[self.label]

        if module_has_submodule(self.module, MODELS_MODULE_NAME):
            models_module_name = "%s.%s" % (self.name, MODELS_MODULE_NAME)
            self.models_module = import_module(models_module_name)

In summary, Django's model detection process leverages metaclasses, specifically the __new__ method, to register model classes during import time.

When you execute a Django command, such as makemigrations or migrate, Django uses django.setup() to import all the models from the models.py files of your installed apps. This is further utilised in various checks and actions performed by django for generating or executing migration files. You can see an example from the makemigrations command below.

# django/django/core/management/commands/makemigrations.py
class Command(BaseCommand):
    help = "Creates new migration(s) for apps."
    ...
    @no_translations
    def handle(self, *app_labels, **options):
        ...
        ...
        for alias in sorted(aliases_to_check):
            connection = connections[alias]
            if connection.settings_dict["ENGINE"] != "django.db.backends.dummy" and any(
                # At least one model must be migrated to the database.
                router.allow_migrate(
                    connection.alias, app_label, model_name=model._meta.object_name
                )
                for app_label in consistency_check_labels
                for model in apps.get_app_config(app_label).get_models()
            ):
            ...