Categories
Python

Understanding Decorators in Python

Many beginners seem to take the concept of decorators as a fairly advanced and complex topic. It’s advanced alright but it probably is much simpler than you think.

Decorators Explained

Let’s say, we have a function which returns a message. But we want to also return the time with the message. So what can we do? We can modify the function’s source code to add the time with the message. But what if we can’t or don’t want to modify the source code but still want to extend/transform the functionality?

In that case, we can wrap it within another function, something like this:

from datetime import datetime


def greet(name):
    return "Greetings, {}!".format(name)


def time_wrapper(fn):
    def new_function(*args, **kwargs):
        msg = fn(*args, **kwargs)
        new_msg = "Time: {} {} ".format(datetime.now(), msg)

        return new_msg

    return new_function


greet = time_wrapper(greet)

print(greet("masnun"))

Here, `greet` was our original function, which only returns a message but no time with it. So we be clever and write a wrapper – `time_wrapper`. This wrapper function takes a function as it’s argument and returns the `new_function` instead. This new function, when invoked, can access the original function we passed, get the message out and then add the time to it.

The interesting bit is here – `greet = time_wrapper(greet)`. We’re passing `greet` to `time_wrapper`. The `time_wrapper` function returns the `new_function`. So `greet` now points to the `new_function`. When we call `greet`, we actually call that function.

By definition, a Decorator is a `callable`s which takes a `callable` and returns a `callable`. A `callable` can be a few things but let’s not worry about that right now. In most cases, a decorator just takes a function, wraps it and returns the wrapped function. The wrapped function can access a reference to our original function and call it as necessary. In our case `time_wrapper` is the decorator function which takes the `greet` function and returns the `new_function`.

The `@` decorator syntax

But you might be wondering – “I see a lot of `@` symbols while reading on decorators, how can there be a decorator without the `@`?”. Well, before PEP 0318, we used to write decorators like that. But soon the wise people of the Python community realized that it would be a good idea to have a nicer syntax for decorators. So we got the `@`. So how does the `@` work?

@decorator_callable
def awesome_func():
    return True

# Equivalent to:

awesome_func = decorator_callable( awesome_func )

So when we add a `callable` name prepended with a `@` on top of a function, that function is passed to that callable. The return value from that callable becomes the new value of that function.

Writing our own decorators

Let’s say we want to write a decorator which will take a function and print the current time every time the function is executed. Let’s call our function `timed`. This function will accept a parameter `fn` which is the function we wrap. Since we need to return a function from the `timed` function, we need to define that function too.

from datetime import datetime

def timed(fn):
    def wrapped():
        print("Current time: {}".format(datetime.now()))
        return fn()

    return wrapped

In this example, the `timed` function takes the `fn` function and returns the `wrapped` function. So by definition it’s a decorator. Within the `wrapped` function, we’re first `print`ing out the current time. And then we’re invoking the `fn()` function. After the decorator is applied, this `wrapped` function becomes the new `fn`. So when we call `fn`, we’re actually calling `wrapped`.

Let’s see example of this decorator:

from time import sleep
from datetime import datetime


def timed(fn):
    def wrapped():
        print("Current time: {}".format(datetime.now()))
        return fn()

    return wrapped


@timed
def hello():
    print("Hello world!")


for x in range(5):
    hello()
    sleep(10)

With the `@timed` decorator applied to `hello`, this happens: `hello = timed(hello)`, `hello` now points to the `wrapped` function returned by `timed`. Inside the for loop, every time we call, hello, it’s no longer the original hello function but the `wrapped` function. The wrapped function calls the copy of the original `hello` from it’s parent scope.

Two things you might have noticed – it is possible to nest functions and when we nest a function within a function, the inner function can access the parent scope too. You can learn more about the scope by reading on `closure`.

Decorator Parameters

Decorators can take parameters too. Like this:

@sleeper(10)               # sleep for 10 secs 
def say_hello(name):
    print("Hello {}!".format(name))

When a decorator takes a parameter, it’s executed like:

say_hello = sleeper(4)(say_hello)

As we can see, it gets a level deeper. Here `sleeper` has to take the parameter and return the actual decorator function which will transform our `say_hello` function.

from time import sleep

def sleeper(secs):
    def decorator(fn):
        def wrapped(*args, **kwargs):
            sleep(secs)
            fn(*args, **kwargs)

        return wrapped

    return decorator

In this case, `sleeper(4)` returns the `decorator` function. We pass `say_hello` to the decorator. The decorator wraps it inside the `wrapped` function and returns `wrapped`. So finally, `say_hello` is actually the `wrapped` function which gets `fn` and `secs` from the closure.

Chaining Decorators

We can chain multiple decorators. Like this:

@sleeper(10)
@sleeper(5)
def say_hello(name):
    print("Hello {}!".format(name))


# This is equivalent to: 

say_hello = sleeper(10)(sleeper(5)(say_hello))

The bottom most one gets executed first, then the returned function is passed to the decorator on top of that one. This way the chain of execution goes from bottom to top.

Using Classes as Decorators

In our previous examples, we have only focused on functions, but in Python, any callables can be used as decorator. That means we can uses Classes too. Let’s first see an example:

from time import sleep
from datetime import datetime


class Sleeper:
    def __init__(self, secs):
        self.__secs = secs

    def __call__(self, fn):
        def wrapped(*args, **kwargs):
            sleep(self.__secs)
            return fn(*args, **kwargs)

        return wrapped


@Sleeper(5)
def say_hello(name):
    print("Hello {}, it is now: {}".format(name, datetime.now()))

for x in range(5):
    say_hello("masnun")

When we’re using the `Sleeper` decorator, we are getting the parameter `5` to the constructor. We are storing it in an instance variable. The constructor returns an object instance, when we call it, it gets the function and returns a decorated, wrapped function.

This is just like before, `say_hello = Sleeper(5)(say_hello)`. The first call is the constructor. The second call is made to the `__call__` magic method.

Decorating Class and Class Methods

We can decorate any callables, so here’s an example where we’re decorating a Class to forcefully convert the `age` argument to `int`.

def int_age(fn):
    def wrapped(**kwargs):
        kwargs['age'] = int(kwargs.get('age'))
        return fn(**kwargs)

    return wrapped


@int_age
class Person:
    def __init__(self, age):
        self.age = age

    def print_age(self):
        print(self.age)


p = Person(age='12')
p.print_age()

We can decorate the methods as well. If you know Python’s OOP model well, you probably have already came across the `@property` decorator. Or the `@classmethod` and `@staticmethod` decorators. These decorate methods.

3 replies on “Understanding Decorators in Python”

Using decorators may be easy, but implementing them correctly if you want them to behave properly as far as preserving introspection and descriptor behaviour is far from easy. For lots of information about the problem, read the blog series:

http://blog.dscpl.com.au/p/decorators-and-monkey-patching.html

As far as getting it right, the wrapt package gets closer than any other package available.

http://wrapt.readthedocs.org

You may like to write another blog post but show your same examples but using wrapt. 🙂

Comments are closed.