OOP III: Advanced Concepts

True dynamism

This is our third lecture on OOP. Here, we treat some concepts that are less of central importance than in the other lectures on OOP, so the focus is more on knowing these things exist, rather than working with them ourselves.

The __new__ method

Object creation vs. initialization

We’ve seen that __init__ initializes a new object, but how is the object actually created? That’s the job of the lesser-known __new__ method.

While we often call __init__ the “constructor,” __new__ is the true constructor that creates the object from nothing. It’s a class method that gets called before __init__:

class Purchase:
    def __new__(cls, name, price):
        print(f'__new__ called: creating {name}')
        return super().__new__(cls)
    
    def __init__(self, name, price):
        print(f'__init__ called: initializing {name}')
        self.name = name
        self.price = price
    
    def __repr__(self):
        return f'Purchase({self.name}, {self.price})'

laptop = Purchase('laptop', 1200)
print(laptop)

__new__ called: creating laptop
__init__ called: initializing laptop
Purchase(laptop, 1200)

Notice that:

1. __new__ receives the same arguments as __init__
2. __new__ must return the created object
3. __new__ is called before __init__

Controlling object creation

You can use __new__ to control how objects are created. Here’s a singleton pattern (only one instance allowed):

class DatabaseConnection:
    _instance = None
    
    def __new__(cls, *args, **kwargs):
        if cls._instance is None:
            print('Creating new database connection')
            cls._instance = super().__new__(cls)
        else:
            print('Reusing existing database connection')
        return cls._instance
    
    def __init__(self, host='localhost'):
        # Be careful: __init__ gets called every time!
        if not hasattr(self, 'host'):  # Only initialize once
            print(f'Initializing connection to {host}')
            self.host = host

db1 = DatabaseConnection('server1')
db2 = DatabaseConnection('server2')  # Same instance!
print(f'db1 is db2: {db1 is db2}')
print(f'Host: {db1.host}')  # Still 'server1'

Creating new database connection
Initializing connection to server1
Reusing existing database connection
db1 is db2: True
Host: server1

Here’s another example: a class that prevents duplicate objects for the same name:

class Person:
    _instances = {}
    
    def __new__(cls, name):
        if name not in cls._instances:
            print(f'Creating new person: {name}')
            instance = super().__new__(cls)
            cls._instances[name] = instance
        else:
            print(f'Returning existing person: {name}')
        return cls._instances[name]
    
    def __init__(self, name):
        if not hasattr(self, 'name'):  # Only initialize once
            self.name = name
            self.friends = []

alice1 = Person('Alice')
alice2 = Person('Alice')
bob = Person('Bob')

print(f'alice1 is alice2: {alice1 is alice2}')
print(f'alice1 is bob: {alice1 is bob}')

Creating new person: Alice
Returning existing person: Alice
Creating new person: Bob
alice1 is alice2: True
alice1 is bob: False

Warning

Be careful with __init__ when using __new__ patterns. Since __init__ is called every time, you might reinitialize your object unexpectedly!

Exercise

Create a ConfigManager class that:

1. Only allows one instance per config file name
2. Stores configuration as a dictionary
3. Provides a method to get/set config values

Test it:

config1 = ConfigManager('app.conf')
config2 = ConfigManager('app.conf')  # Should be same instance
config3 = ConfigManager('db.conf')   # Should be different instance

config1.set('debug', True)
print(config2.get('debug'))  # Should print True

Dynamic attribute access

__getattr__ and __setattr__

Python provides hooks to customize attribute access. The terminology: x.attr means “attr is an attribute of object x” (can be data or methods).

__getattr__ - handling missing attributes

__getattr__ is called when an attribute can’t be found through normal lookup:

class FlexibleObject:
    def __init__(self):
        self.existing_attr = "I exist!"
    
    def __getattr__(self, name):
        if name.startswith('get_'):
            attr_name = name[4:]  # Remove 'get_' prefix
            return lambda: f"Getting {attr_name}"
        elif name in 'abc':
            return name.upper()
        else:
            raise AttributeError(f"'{type(self).__name__}' has no attribute '{name}'")

obj = FlexibleObject()

print(obj.existing_attr)  # Normal attribute lookup
print(obj.a)              # Triggers __getattr__
print(obj.get_username())  # Creates a function dynamically
# print(obj.nonexistent)  # Would raise AttributeError

I exist!
A
Getting username

__setattr__ - controlling attribute setting

__setattr__ is called for every attribute assignment:

class ValidatedObject:
    def __setattr__(self, name, value):
        if name.startswith('_'):
            # Allow private attributes
            super().__setattr__(name, value)
        elif isinstance(value, str) and len(value) < 3:
            raise ValueError(f"String attributes must be at least 3 characters")
        elif name in ['x', 'y'] and not isinstance(value, (int, float)):
            raise TypeError(f"Attribute {name} must be numeric")
        else:
            super().__setattr__(name, value)

obj = ValidatedObject()
obj.x = 10        # OK
obj.name = "Alice"   # OK
obj._private = "hidden"  # OK

try:
    obj.name = "Al"  # Too short!
except ValueError as e:
    print(f"Error: {e}")

try:
    obj.x = "not a number"  # Wrong type!
except TypeError as e:
    print(f"Error: {e}")

Error: String attributes must be at least 3 characters
Error: Attribute x must be numeric

A practical example: AttributeDict

Here’s a dict that allows attribute-style access:

class AttributeDict:
    def __init__(self, **kwargs):
        # Use object.__setattr__ to avoid infinite recursion
        object.__setattr__(self, '_data', kwargs)
    
    def __getattr__(self, name):
        if name in self._data:
            return self._data[name]
        raise AttributeError(f"No attribute '{name}'")
    
    def __setattr__(self, name, value):
        self._data[name] = value
    
    def __repr__(self):
        return f"AttributeDict({self._data})"

config = AttributeDict(debug=True, host='localhost', port=8080)
print(config)
print(f"Debug mode: {config.debug}")
print(f"Server: {config.host}:{config.port}")

config.timeout = 30
print(f"Added timeout: {config.timeout}")

AttributeDict({'debug': True, 'host': 'localhost', 'port': 8080})
Debug mode: True
Server: localhost:8080
Added timeout: 30

Exercise

Create a LoggingObject class that:

1. Logs every attribute access with __getattr__
2. Logs every attribute modification with __setattr__
3. Stores actual data in a private dictionary
4. Has a method to show the access log

Test it with various attribute operations.

The descriptor protocol

Understanding descriptors

Descriptors are a powerful way to customize attribute access. A descriptor is an object with __get__, __set__, or __delete__ methods that controls access to an attribute.

class LoggingDescriptor:
    def __init__(self, name):
        self.name = name
        self.data = {}  # Store data per instance
    
    def __get__(self, instance, owner):
        if instance is None:
            return self  # Called on class
        print(f"Getting {self.name}")
        return self.data.get(id(instance), None)
    
    def __set__(self, instance, value):
        print(f"Setting {self.name} to {value}")
        self.data[id(instance)] = value
    
    def __delete__(self, instance):
        print(f"Deleting {self.name}")
        self.data.pop(id(instance), None)

class Person:
    name = LoggingDescriptor('name')
    age = LoggingDescriptor('age')
    
    def __init__(self, name, age):
        self.name = name  # Triggers __set__
        self.age = age    # Triggers __set__

person = Person("Alice", 30)
print(f"Person's name: {person.name}")  # Triggers __get__
person.age = 31  # Triggers __set__
print(f"Person's age: {person.age}")   # Triggers __get__

Setting name to Alice
Setting age to 30
Getting name
Person's name: Alice
Setting age to 31
Getting age
Person's age: 31

Practical example: Temperature converter

class TemperatureDescriptor:
    def __init__(self, unit):
        self.unit = unit
        self.kelvin_data = {}  # Store in Kelvin internally
    
    def __get__(self, instance, owner):
        if instance is None:
            return self
        
        kelvin = self.kelvin_data.get(id(instance), 0)
        
        if self.unit == 'K':
            return kelvin
        elif self.unit == 'C':
            return kelvin - 273.15
        elif self.unit == 'F':
            return (kelvin - 273.15) * 9/5 + 32
    
    def __set__(self, instance, value):
        if self.unit == 'K':
            kelvin = value
        elif self.unit == 'C':
            kelvin = value + 273.15
        elif self.unit == 'F':
            kelvin = (value - 32) * 5/9 + 273.15
        
        self.kelvin_data[id(instance)] = kelvin

class Temperature:
    kelvin = TemperatureDescriptor('K')
    celsius = TemperatureDescriptor('C')
    fahrenheit = TemperatureDescriptor('F')
    
    def __init__(self, celsius=0):
        self.celsius = celsius

temp = Temperature(25)  # 25°C
print(f"Temperature: {temp.celsius}°C = {temp.fahrenheit}°F = {temp.kelvin}K")

temp.fahrenheit = 100  # Set to 100°F
print(f"Temperature: {temp.celsius}°C = {temp.fahrenheit}°F = {temp.kelvin}K")

Temperature: 25.0°C = -459.66999999999996°F = 0K
Temperature: 25.0°C = 99.99999999999999°F = 0K

Types of descriptors

• Non-data descriptors: Only have __get__ (like functions/methods)
• Data descriptors: Have __set__ and/or __delete__ (like properties)

Many Python features use descriptors behind the scenes: - Properties (@property) - Methods (bound/unbound) - Class methods (@classmethod) - Static methods (@staticmethod)

import types

def show_descriptor_info(cls):
    for name, attr in vars(cls).items():
        if hasattr(attr, '__get__'):
            desc_type = "data" if hasattr(attr, '__set__') or hasattr(attr, '__delete__') else "non-data"
            print(f"{name}: {type(attr).__name__} ({desc_type} descriptor)")

class Example:
    class_var = "I'm just a variable"
    
    @property
    def prop(self):
        return "I'm a property"
    
    @classmethod
    def class_method(cls):
        return "I'm a class method"
    
    def regular_method(self):
        return "I'm a regular method"

show_descriptor_info(Example)

prop: property (data descriptor)
class_method: classmethod (non-data descriptor)
regular_method: function (non-data descriptor)
__dict__: getset_descriptor (data descriptor)
__weakref__: getset_descriptor (data descriptor)

Exercise

Create a ValidatedAttribute descriptor that: 1. Takes a validation function in its constructor 2. Only allows values that pass validation 3. Raises ValueError for invalid values

Use it to create a Person class with: - name: must be a non-empty string - age: must be between 0 and 150 - email: must contain ‘@’

Test with valid and invalid values.

Inheritance design patterns

We will see more on these topics in our lectures on Architecture.

Classes designed to be subclassed

Sometimes you design a base class that’s meant to be extended, providing a template with some methods implemented and others left for subclasses:

class DataProcessor:
    """Base class for processing data with a common workflow."""
    
    def process(self, data):
        """Main processing workflow - don't override this."""
        print("Starting data processing...")
        
        validated = self.validate(data)
        if not validated:
            raise ValueError("Data validation failed")
        
        transformed = self.transform(data)
        result = self.analyze(transformed)
        
        print("Processing complete!")
        return result
    
    def validate(self, data):
        """Override this in subclasses."""
        raise NotImplementedError("Subclasses must implement validate()")
    
    def transform(self, data):
        """Override this in subclasses."""
        raise NotImplementedError("Subclasses must implement transform()")
    
    def analyze(self, data):
        """Override this in subclasses."""
        raise NotImplementedError("Subclasses must implement analyze()")

class NumberProcessor(DataProcessor):
    def validate(self, data):
        return isinstance(data, list) and all(isinstance(x, (int, float)) for x in data)
    
    def transform(self, data):
        return [x * 2 for x in data]  # Double all numbers
    
    def analyze(self, data):
        return {
            'sum': sum(data),
            'average': sum(data) / len(data),
            'count': len(data)
        }

processor = NumberProcessor()
result = processor.process([1, 2, 3, 4, 5])
print(result)

Starting data processing...
Processing complete!
{'sum': 30, 'average': 6.0, 'count': 5}

Abstract base classes with abc

Python provides the abc module for creating formal abstract base classes:

from abc import ABC, abstractmethod

class Shape(ABC):
    """Abstract base class for shapes."""
    
    @abstractmethod
    def area(self):
        """Calculate the area of the shape."""
        pass
    
    @abstractmethod
    def perimeter(self):
        """Calculate the perimeter of the shape."""
        pass
    
    def describe(self):
        """Concrete method that uses abstract methods."""
        return f"This shape has area {self.area():.2f} and perimeter {self.perimeter():.2f}"

class Rectangle(Shape):
    def __init__(self, width, height):
        self.width = width
        self.height = height
    
    def area(self):
        return self.width * self.height
    
    def perimeter(self):
        return 2 * (self.width + self.height)

class Circle(Shape):
    def __init__(self, radius):
        self.radius = radius
    
    def area(self):
        return 3.14159 * self.radius ** 2
    
    def perimeter(self):
        return 2 * 3.14159 * self.radius

# Can't instantiate abstract class
try:
    shape = Shape()
except TypeError as e:
    print(f"Error: {e}")

# Can instantiate concrete classes
rect = Rectangle(5, 3)
circle = Circle(2)

print(rect.describe())
print(circle.describe())

Error: Can't instantiate abstract class Shape without an implementation for abstract methods 'area', 'perimeter'
This shape has area 15.00 and perimeter 16.00
This shape has area 12.57 and perimeter 12.57

Mixin classes

Mixins are classes designed to be combined with other classes via multiple inheritance:

class TimestampMixin:
    """Mixin to add timestamp functionality."""
    
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        import time
        self._created_at = time.time()
    
    def get_age(self):
        import time
        return time.time() - self._created_at

class ReprMixin:
    """Mixin to provide a nice string representation."""
    
    def __repr__(self):
        attrs = []
        for key, value in self.__dict__.items():
            if not key.startswith('_'):
                attrs.append(f"{key}={value}")
        return f"{self.__class__.__name__}({', '.join(attrs)})"

class Product(TimestampMixin, ReprMixin):
    def __init__(self, name, price):
        super().__init__()
        self.name = name
        self.price = price

product = Product("Laptop", 999)
print(product)  # Uses ReprMixin
import time
time.sleep(1)
print(f"Product age: {product.get_age():.2f} seconds")  # Uses TimestampMixin

Product(name=Laptop, price=999)
Product age: 1.00 seconds

Exercise

1. Abstract Shape Factory: Create an abstract ShapeFactory class with an abstract method create_shape(). Then create RectangleFactory and CircleFactory subclasses.

2. Logging Mixin: Create a LoggingMixin that:

   • Adds a log() method to record messages with timestamps
   • Adds a get_logs() method to retrieve all messages
   • Use it with other classes to add logging capability

3. Template Method Pattern: Create a GameCharacter base class that defines a take_turn() method calling abstract methods choose_action(), execute_action(), and end_turn(). Create concrete characters like Warrior and Mage.

Advanced attribute access patterns

__getattribute__ vs __getattr__

There’s a difference between these two methods:

• __getattr__: Only called when attribute lookup fails
• __getattribute__: Called for every attribute access

class Monitored:
    def __init__(self):
        self.data = {'x': 10, 'y': 20}
    
    def __getattribute__(self, name):
        print(f"Accessing attribute: {name}")
        return super().__getattribute__(name)
    
    def __getattr__(self, name):
        print(f"Attribute {name} not found, checking data dict")
        return self.data.get(name, f"No attribute {name}")

obj = Monitored()
print(f"obj.data: {obj.data}")  # Triggers __getattribute__
print(f"obj.x: {obj.x}")        # Triggers both __getattribute__ and __getattr__
print(f"obj.z: {obj.z}")        # Triggers both __getattribute__ and __getattr__

Accessing attribute: data
obj.data: {'x': 10, 'y': 20}
Accessing attribute: x
Attribute x not found, checking data dict
Accessing attribute: data
obj.x: 10
Accessing attribute: z
Attribute z not found, checking data dict
Accessing attribute: data
obj.z: No attribute z

Additional resources

• Python Data Model Documentation
• Descriptor Protocol Guide
• Abstract Base Classes
• Python’s Method Resolution Order