Python __new__ vs __init__

The constructor in Python is composed of two methods __new__ and __init__, executed one after the other. __new__ creates an instance, which is then passed to __init__.

Let’s review the properties and use cases of this somewhat exotic part of Python.

__new__ basic information

First of all, __new__ is a static method, therefore its first argument is a reference to the class it is called on, widely named cls.

The remaining arguments are due to the class call itself, so if __init__ takes arguments, you must define the same parameters in __new__. Because of this, the next block of code will throw an exception:

TypeError: new() takes 1 positional argument but 2 were given

class Point:
    def __new__(cls):
        ...

    def __init__(self, x):
        self.x = x

Point(5)

The value that __new__ returns is the value returned from calling the class, which may lead to messy code, but we will take it later on.

class Dummy:
    def __new__(cls):
        return 1

print(Dummy() == 1)

# Outputs:
# True

So in consequence, if __new__ does not return an instance of the class on which it is called, __init__ will not be called.

class Dummy:
    ...

class MyClass:
    def __new__(cls):
        return Dummy()

    def __init__(self):
        print('Will not be printed')

print(isinstance(MyClass(), Dummy))

# Outputs:
# True

Instance type based on constructor arguments

The first example smacks of anti-pattern because the returned value is of a different type than the class that is called. Let’s focus on the behavior of the methods for now.

If the sides of the rectangle are equal, then isn’t it better to return a square?

class Square:
    def __init__(self, side_length):
        self.side_length = side_length


class Rectangle:
    def __new__(cls, width: float, height: float):
        if width == height:
            return Square(side_length=width)

        return object.__new__(Rectangle)

    def __init__(self, width: float, height: float):
        self.width = width
        self.height = height

r1 = Rectangle(2, 3)
r2 = Rectangle(2, 2)

print(type(r1))
print(type(r2))

# Outputs:
# <class '__main__.Rectangle'>
# <class '__main__.Square'>

Usage in the Singleton design pattern

The purpose of the Singleton pattern is to limit the number of instances created. There are plenty of implementations of it using mostly __init__.

One of the most interesting implementations of this pattern, drawing on Python’s Simple is better than complex credo, involves immediately deleting a class after creating an instance. Of course, this has its drawback, as it does not allow for lazy loading. But it is short and simple:

class Singleton:
    def __init__(self, *args, **kwargs):
        pass


singleton_instance = Singleton()
del Singleton

Back on topic, here’s an implementation of Singleton using __new__

class Singleton:
    _instance = None  # Keep instance reference

    def __new__(cls, *args, **kwargs):
        if not cls._instance:
            cls._instance = super().__new__(cls)
        return cls._instance


s1 = Singleton()
s2 = Singleton()

print(s1 == s2) # → True

Inheritance caveats

The previous examples can be implemented in other ways without being aware of __new__ itself. But now I will present you with situations that __new__ is irreplaceable.

Suppose that SubClass inherits from BaseClass:

class BaseClass:
    pass

class SubClass(BaseClass):
    pass

isinstance(SubClass(), BaseClass) # True

Next, you want to get information about whether the call to the BaseClass instance comes from itself or an inheriting class.

class BaseClass:
    def __new__(cls):
        obj = super(BaseClass, cls).__new__(cls)
        obj._from_base_class = type(obj) == BaseClass
        return obj


class SubClass(BaseClass):
    ...


base_instance = BaseClass()
sub_instance = SubClass()

print(base_instance._from_base_class) # True
print(sub_instance._from_base_class) # False

Why can’t this be implemented using __init__? Because the self in __init__, is already an instance. In other words, when you are in, __init__ it is too late to track the class which was used to create an instance.

inherit from immutable types

__new__ allows us to modify the returned value. If we want to modify the creation of immutable types, __init__ is useless, because in this part of the constructor it is too late, we got already immutable instance. That’s why we need the __new__ method.

So let’s create a PositiveNumberTuple class that meets the requirements:

1. Inherit from tuple
2. store only float values
3. filters out values smaller than zero
4. store information about how many values were skipped

class PositiveNumberTuple(tuple): # 1
    def __new__(cls, *numbers):
        skipped_values_count = 0 # 4
        positive_numbers = []
        for x in numbers:
            if x >= 0: # 2, 3
                positive_numbers.append(x)
            else:
                skipped_values_count += 1

        instance = super().__new__(cls, tuple(positive_numbers))

        instance.skipped_values_count = skipped_values_count

        return instance


positive_ints_tuple = PositiveNumberTuple(-2, -1, 0, 1, 2)

print(positive_ints_tuple)  # -> (0, 1, 2)
print(type(positive_ints_tuple))  # -> <class '__main__.PositiveNumberTuple'> 
print(positive_ints_tuple.skipped_values_count)  # -> 2

Conclusions

In summary, it is easy to abuse __new__ usage, its applications of which cannot be replaced in any other, more common way is very narrow. However, as you have seen, it exists for a reason.