Decoding Python Magic -> Descriptor Protocol

Let’s Decode Mighty Descriptor Protocol

Greetings, everyone! Today, we’re delving into one of Python’s lesser-known yet compelling concepts: the Descriptor Protocol. Descriptors empower objects to tailor attribute lookup, storage, and deletion, offering a level of customization that goes beyond the typical class attributes we’re accustomed to.

Image Credit Unsplash

Basic Examples

Let’s create a class to store programming languages along with their creators’ names.

class CCreator:
    def __get__(self, obj, objtype=None):
        print("calling CCreator.__get__")
        return "Dennis Ritchie"


class LanguageCreators:
    c = CCreator()
    python = "Guido van Rossum"

c = LanguageCreators()
print(c.c)
print(c.python)

calling CCreator.__get__
Dennis Ritchie
Guido van Rossum

• Descriptors allow you to define behavior for attribute access, including getting, setting, and deleting. Here, when accessing c in LanguageCreators, the __get__ method of CCreator is invoked.

Retrieve Folder Size

Let’s create a utility class that retrieves the size of a folder.

import os


class SizeGetter:

    def __get__(self, obj, objtype=None):
        return len(os.listdir(obj.filename))


class DirectorySize:
    size = SizeGetter()

    def __init__(self, filename):
        self.filename = filename


d = DirectorySize("/home/rahul/python/fastapi")
print(d.size)
os.remove("/home/rahul/python/fastapi/file.txt")
print(d.size)

13
12

• d.size: This attribute is not static; its value is calculated at runtime every time you access d.size.

The Requirements to be a Descriptor

A descriptor is what we call any object that defines __get__(), __set__(), or __delete__(). Optionally, descriptors can have a __set_name__() method.

Descriptors only work when used as class variables. When put in instances, they have no effect.

object defines __set__() or __delete__(), it is considered a data descriptor. Descriptors that only define __get__() are called non-data descriptors

Age Validator

Let’s create a class that validates age before it is set.

class AgeValidator:

    def __get__(self, obj, objtype=None):
        return obj._age

    def __set__(self, obj, value):
        if value < 0 or not isinstance(value, int) or value > 110 or value < 18:
            raise ValueError("Age should be a positive integer between 18 and 110")
        obj._age = value


class Person:
    age = AgeValidator()

    def __init__(self):
        self._age = None


d = Person()
d.age = 30
print(d.age)
d.age = -1

ValueError: Age should be a positive integer between 18 and 110

• a.age = -1 : Invokes __set__ method where obj is d and value is -1.

Let’s implement the logic for resetting a Person’s age.

class AgeValidator:

    def __get__(self, obj, objtype=None):
        return obj._age

    def __set__(self, obj, value):
        if value < 0 or not isinstance(value, int) or value > 110 or value < 18:
            raise ValueError("Age should be a positive integer between 18 and 110")
        obj._age = value
    
    def __delete__(self, obj):
        obj._age = None


class Person:
    age = AgeValidator()

    def __init__(self):
        self._age = None


d = Person()
d.age = 30
print(d.age)
del d.age
print(d.age)

# OUTPUT
# 30
# None

Practical Implementation (A Useful Validator)

Let’s create a validator with the following options:

1. Validate age with a minimum and maximum range.
2. Validate a string with a minimum and maximum length.

from abc import ABC, abstractmethod


class Validator(ABC):

    def __set_name__(self, owner, name):
        self.private_name = "_" + name

    def __get__(self, obj, objtype=None):
        return obj.__dict__[self.private_name]

    def __set__(self, obj, value):
        self.validate(value)
        setattr(obj, self.private_name, value)

    @abstractmethod
    def validate(self, value):
        pass


class PositiveNumber(Validator):
    def __init__(self, minvalue=None, maxvalue=None):
        self.minvalue = minvalue
        self.maxvalue = maxvalue

    def validate(self, value):
        if not isinstance(value, int):
            raise ValueError(f"{value!r} should be an integer")
        if value < 0:
            raise ValueError(f"{value!r} should be an positive integer")
        if self.minvalue is not None and value < self.minvalue:
            raise ValueError(f"{value!r} should be greater than {self.minvalue - 1}")

        if self.maxvalue is not None and value > self.maxvalue:
            raise ValueError(f"{value!r} should be less than {self.maxvalue + 1}")


class StringValidator(Validator):
    def __init__(self, minlen=None, maxlen=None):
        self.minlen = minlen
        self.maxlen = maxlen

    def validate(self, value):
        if not isinstance(value, str):
            raise ValueError(f"value {value!r} be a string")
        if self.minlen is not None and len(value) < self.minlen:
            raise ValueError(f"{value!r} should be longer than {self.minlen - 1}")
        if self.maxlen is not None and len(value) > self.maxlen:
            raise ValueError(f"{value!r} should be shorter than {self.maxlen + 1}")


class Person:
    name = StringValidator(minlen=2, maxlen=10)
    age = PositiveNumber(minvalue=18, maxvalue=110)

    def __init__(self, name, age):
        self.name = name
        self.age = age


p = Person(name="JAY", age=20)
print(p.name)

# OUTPUT
# JAY

• __set_name__ : When a class uses descriptors, it can inform each descriptor about which variable name was used.
• In this example, the Person class has two descriptor instances, name and age. When the Person class is defined, it makes a callback to __set_name__() to store the descriptor name to self._private.

Using property decorator to create descriptors.

The property decorator, which is commonly used as @property, is itself implemented as a descriptor. It can be used to create descriptors cleanly and concisely.

Let’s Re-Create AgeValidator

class Person:
    def __init__(self):
        self._age = None

    def get_age(self):
        return self._age

    def set_age(self, value):
        if value < 0 or not isinstance(value, int) or value > 110 or value < 18:
            raise ValueError("Age should be a positive integer between 18 and 110")
        self._age = value

    def del_age(self):
        self._age = None

    age = property(get_age, set_age, del_age)

property(fget=None, fset=None, fdel=None, doc=None) -> property

But more elegantly

class Person:
    def __init__(self):
        self._age = None

    @property
    def age(self):
        return self._age

    @age.setter
    def age(self, value):
        if value < 0 or not isinstance(value, int) or value > 110 or value < 18:
            raise ValueError("Age should be a positive integer between 18 and 110")
        self._age = value

    @age.deleter
    def age(self):
        self._age = None

Practical Use Cases

Django Use Descriptors to define Models and Generate SQL.

from django.db import models


class Person(models.Model):
    name = models.CharField(max_length=100)
    age = models.IntegerField()

    class Meta:
        db_table = "person"

Standard Library Uses

Common tools like classmethod(), staticmethod(), property(), and functools.cached_property() are all implemented as descriptors.

The use cases illustrated above are some of the common scenarios where descriptors are employed. However, descriptors are extensively used in both third-party packages and the standard library.

Conclusion

The Descriptor Protocol in Python provides a powerful mechanism for customizing attribute access, enabling objects to define their behavior when attributes are accessed, set, or deleted. By implementing the __get__, __set__, and __delete__ methods in a descriptor class, developers can control how attributes are managed, allowing for validation, lazy initialization, data conversion, and more.

In practical terms, descriptors are used to validate input data, create property-like behavior, enforce access control, and map Python objects to database columns in ORMs like Django.

If you enjoy this Article, Follow Rahul Beniwal for more.

Similar articles by me.

Decoding Python Magic -> __getitem__ and __setitem__

Decoding Python Magic : __setattr__

Decoding Python Magic: __getattribute__ and __getattr__

Demystifying Database Partitioning: Impact With Practical Example