In this article we will look at how to create your own recursive serializer in Django without using the Django Rest Framework Sometimes when working with Django, we may have some data that we want to serialize (convert to JSON) but we do not have the option of using the serializers that come with Django Rest Framework. The data can also take the same form for many cases and writing a new serializer for all of those cases can be tedious and repetitive. This sounds like something a utility function that can be used to generalize the logic and be reused where needed, hence this article’s existence.

By the end of the article, we should have a utility function like this:

import json

from somewhere import ModelFoo
from utilities import generic_serializer

instances = ModelFoo.objects.filter(some_filter)
serialized_instances = generic_serializer(instance, i_fields=['bar'], i_models=['baz'])

with open('Foo_data.json', 'w') as fd:
    json.dump(serialized_instances, fd, default=str, indent=2)

Defining the models

To explain things better, let us use an example. Suppose we have a Django application that stores data for a chain of car dealerships. Each dealership has cars that customers go to one of them to buy a car. The example models.py is shown below:

# models.py
from django.db import models

class Dealership(models.Model):
    name = models.CharField(max_length=20) 
    street_name = models.CharField(max_length=20) 
    zip_code = models.CharField(max_length=6) 

    def __str__(self):
        return self.name

class Car(models.Model):
    name = models.CharField(max_length=20) 
    brand = models.CharField(max_length=20) 
    year = models.CharField(max_length=20) 
    price = models.DecimalField(max_digits=7, decimal_places=2)
    dealership = models.ForeignKey(Dealership, on_delete=models.CASCADE)

    def __str__(self):
        return f"{self.name} {self.brand}"

Creating the generic serializer

Finding the fields

To start us off the first thing our serializer need to be able to do to create a dict of the fields and values of our given model instance. In this article, we will focus on the Car model instance and try to serialize it. To give us something to work with we create some sample instances:

# backups.py
dealership = Dealership.objects.create(name="LopezCars", street_name="Tarmac", zip_code="90210")
# {"id":1, "name":"LopezCars", "street_name": "Tarmac", "zip_code": "90210"}

car = Car.objects.create(name="Camión ", brand="Suave", year="2020", price="2283", dealership=1)
# {"id":1, "name":"Camión", "brand":"Suave", "year":"2020", "price":Decimal('2283.00'), "dealership": 1}

The simplest way to convert a model instance to a dict in django is to use the model_to_dict function that is built into django. To use it, simply import it and pass a model instance to it. It supports include all or exclude all but the specified fields.

>>> from django.forms import model_to_dict
>>> from .backups import car
>>>
>>> model_to_dict(car)
>>> {"name":"Camión", "model":"Suave", "year":"2020", "price":Decimal('2283.00'), "dealership": 1}
>>>
>>> model_to_dict(car, fields=["brand", "price"])
>>> {"model":"Suave", "price":Decimal('2283.00')}
>>>
>>> model_to_dict(car, exclude=["brand", "price"])
>>> {"name":"Camión", "year":"2020", "dealership": 1}

While model_to_dict is very useful for a simple model, it is not powerful enough for what we may need, like recursively fetching related models. Django models have the _meta API which is an instance an django.db.models.options.Options object that allows us to fetch all the field instances of a model. One of the properties available in _meta is the fields which is a django.utils.datastructures.ImmutableList.

We can use this list to get all fields and construct a dict object out of it like so:

# backups.py
...
fields = {}
for f in car._meta.fields:
    fields[f.name] = getattr(model, f.name)

# if you wish to use a comprehension:
fields = {f.name: getattr(model, f.name) for f in car._meta.fields}

# {"id":1, "name":"Camión", "brand":"Suave", "year":"2020", "price":Decimal('2283.00'), "dealership": <Dealership: LopezCars> }

Looking at the output, you may notice that with this method, we preserve the object Primary Key (id) a.k.a pk and the dealership is not just the id. If we do the same thing for dealership:

# backups.py
...
# if you wish to use a comprehension:
fields = {f.name: getattr(model, f.name) for f in dealership._meta.fields}

# {"id":1, "name":"LopezCars", "street_name": "Tarmac", "zip_code": "90210"}

And now we combine the two:

# backups.py
fields = {f.name: getattr(model, f.name) for f in car._meta.fields}
fields['dealership'] = {f.name: getattr(model, f.name) for f in dealership._meta.fields}

# {"id":1, "name":"Camión", "brand":"Suave", "year":"2020", "price":Decimal('2283.00'), "dealership": {"id":1, "name":"LopezCars", "street_name": "Tarmac", "zip_code": "90210"}}

We have successfully converted the model into a dictionary. This is nice, but it is specific to car so to make it generic using the magic of recursion:

# utilities.py
from django.db.models import Model
def get_fields(model: models.Model, fields: dict = None) -> dict:
    if not fields:
        fields = {}
        for f in model._meta.fields:
            fields[f.name] = getattr(model, f.name)

    for name, value in fields.items():
        if not isinstance(value, model.Model):
            # skip non-relational (ForeignKey) fields
            continue
        fields[name] = getattr(value, "pk") if name in ignore_models else get_fields(value)
    return fields

get_fields(car)

# {"id":1, "name":"Camión", "brand":"Suave", "year":"2020", "price":Decimal('2283.00'), "dealership": {"id":1, "name":"LopezCars", "street_name": "Tarmac", "zip_code": "90210"}}

Now we have a generic function to transform any model into a dict object and that can recurse into the other models within it. If that is all you needed, then you can stop here. We have made a slightly more powerful version of model_to_dict.

Extra Functionality

The get_fields function is fine the way it is and is perfectly usable now. But what if we don’t need the to see the brand field or maybe dealership related model? If we want to get information for more than one car we would have to run the function on both of them. To achieve these things we need to expand it with a few things.

# utilities.py
from django.db.models import Model

def generic_serializer(
    instances: list,
    exclude_models: list = None,
    exclude_fields: list = None,
) -> list:
    def get_fields(model: Model = None, fields: dict = None) -> dict:
        if not fields:
            fields = {}
            for f in model._meta.fields:
                if (
                    not f.is_relation
                    and f.name in exclude_fields
                ):
                    continue
                fields[f.name] = getattr(model, f.name)

        for name, value in fields.items():
            if not isinstance(value, Model):
                continue
            # Replace the excluded related model field with their primary key value
            # Instead of storing the `model.__str__()` value, it shows the model.pk
            # if it is in `exclude_models`
            fields[name] = getattr(value, "pk") if name in exclude_models else get_fields(value)
        return fields

    return [get_fields(instance) for instance in instances]

To check if a field f is a relation field i.e models.ForeignKey field, we can use f.is_relation. For the car model the dealership field is a relational field to Dealership models. Therefor, the value for for dealership.is_relation is True. This allows the exclusion of non-relation field in exclude_field without excluding a relation field with the same name and vice versa. The line if not is instance(value, Model) checks if there there are any relation fields to trigger a recursion.

We are done! (^O^)/ It is a simple function, but it is very useful.

Thanks for taking the time to read this article and I hope it has been useful to you.