Observers

This module provides a decorator function for observing changes in a given property. Internally the decorator is implemented using SQLAlchemy event listeners. Both column properties and relationship properties can be observed.

Property observers can be used for pre-calculating aggregates and automatic real-time data denormalization.

Simple observers

At the heart of the observer extension is the observes() decorator. You mark some property path as being observed and the marked method will get notified when any changes are made to given path.

Consider the following model structure:

class Director(Base):
    __tablename__ = 'director'
    id = sa.Column(sa.Integer, primary_key=True)
    name = sa.Column(sa.String)
    date_of_birth = sa.Column(sa.Date)

class Movie(Base):
    __tablename__ = 'movie'
    id = sa.Column(sa.Integer, primary_key=True)
    name = sa.Column(sa.String)
    director_id = sa.Column(sa.Integer, sa.ForeignKey(Director.id))
    director = sa.orm.relationship(Director, backref='movies')

Now consider we want to show movies in some listing ordered by director id first and movie id secondly. If we have many movies then using joins and ordering by Director.name will be very slow. Here is where denormalization and observes() comes to rescue the day. Let’s add a new column called director_name to Movie which will get automatically copied from associated Director.

from sqlalchemy_utils import observes


class Movie(Base):
    # same as before..
    director_name = sa.Column(sa.String)

    @observes('director')
    def director_observer(self, director):
        self.director_name = director.name

Note

This example could be done much more efficiently using a compound foreign key from director_name, director_id to Director.name, Director.id but for the sake of simplicity we added this as an example.

Observes vs aggregated

observes() and aggregates.aggregated() can be used for similar things. However performance wise you should take the following things into consideration:

• observes() works always inside transaction and deals with objects. If the relationship observer is observing has a large number of objects it’s better to use aggregates.aggregated().
• aggregates.aggregated() always executes one additional query per aggregate so in scenarios where the observed relationship has only a handful of objects it’s better to use observes() instead.

Example 1. Movie with many ratings

Let’s say we have a Movie object with potentially thousands of ratings. In this case we should always use aggregates.aggregated() since iterating through thousands of objects is slow and very memory consuming.

Example 2. Product with denormalized catalog name

Each product belongs to one catalog. Here it is natural to use observes() for data denormalization.

Deeply nested observing

Consider the following model structure where Catalog has many Categories and Category has many Products.

class Catalog(Base):
    __tablename__ = 'catalog'
    id = sa.Column(sa.Integer, primary_key=True)
    product_count = sa.Column(sa.Integer, default=0)

    @observes('categories.products')
    def product_observer(self, products):
        self.product_count = len(products)

    categories = sa.orm.relationship('Category', backref='catalog')

class Category(Base):
    __tablename__ = 'category'
    id = sa.Column(sa.Integer, primary_key=True)
    catalog_id = sa.Column(sa.Integer, sa.ForeignKey('catalog.id'))

    products = sa.orm.relationship('Product', backref='category')

class Product(Base):
    __tablename__ = 'product'
    id = sa.Column(sa.Integer, primary_key=True)
    price = sa.Column(sa.Numeric)

    category_id = sa.Column(sa.Integer, sa.ForeignKey('category.id'))

observes() is smart enough to:

• Notify catalog objects of any changes in associated Product objects
• Notify catalog objects of any changes in Category objects that affect products (for example if Category gets deleted, or a new Category is added to Catalog with any number of Products)

category = Category(
    products=[Product(), Product()]
)
category2 = Category(
    product=[Product()]
)

catalog = Catalog(
    categories=[category, category2]
)
session.add(catalog)
session.commit()
catalog.product_count  # 2

session.delete(category)
session.commit()
catalog.product_count  # 1

sqlalchemy_utils.observer.observes(path, observer=<PropertyObserver>)

Mark method as property observer for the given property path. Inside transaction observer gathers all changes made in given property path and feeds the changed objects to observer-marked method at the before flush phase.

from sqlalchemy_utils import observes


class Catalog(Base):
    __tablename__ = 'catalog'
    id = sa.Column(sa.Integer, primary_key=True)
    category_count = sa.Column(sa.Integer, default=0)

    @observes('categories')
    def category_observer(self, categories):
        self.category_count = len(categories)

class Category(Base):
    __tablename__ = 'category'
    id = sa.Column(sa.Integer, primary_key=True)
    catalog_id = sa.Column(sa.Integer, sa.ForeignKey('catalog.id'))


catalog = Catalog(categories=[Category(), Category()])
session.add(catalog)
session.commit()

catalog.category_count  # 2

Parameters:

• path – Dot-notated property path, eg. ‘categories.products.price’
• observer – PropertyObserver() object