Coroutines in Python

A coroutine is a generator. However, PEP 342 proposed extending generators with several constructs that we'll cover here.

One of the key features of coroutines is the ability to pause a function's execution until data arrives, while preserving its context. The simplest coroutine looks like this:

PythonCopy

def coro():
    val = (yield)
    print(val)

c = coro()  # Объявляем корутину
next(c)  # Просим выполнить метод до первого `yield`
c.send(12)  # Передаём данные в корутину
c.close()  # Завершаем корутину

As a result, the console shows 12 and a GeneratorExit error: the coroutine received the data through the send method, printed it, and finished.

Coroutines can not only yield values and remember where the code stopped, but also wait for new values. For this there is the (yield) construct, which lets a coroutine accept a set of parameters. Since receiving parameters in a coroutine happens in an unusual way, sending them is done through the special send(...) function. At the end you can call the close() method, which stops the coroutine. When you call close(), a GeneratorExit exception is raised, which you can catch and handle gracefully.

Let's look at a more complex example: imagine a method that takes some value, processes it, and returns a result. Let it be a function that calculates how much money will be in your account after N years at a given interest rate. The function takes the annual deposit interest rate and the current account balance.

PythonCopy

import math

def cash_return(deposit: int, percent: float, years: int) -> float:
    value = math.pow(1 + percent / 100, years)
    return round(deposit * value, 2)

Let's find out how much money you'll get after 5 years if the amount on deposit is 1,000,000 rubles and the deposit rate is 5% per year.

PythonCopy

>>> cash_return(1_000_000, 5, 5)
1276281.56

That's quite a decent sum!

Now you want to see how the final amount changes depending on the deposit size. This is where a coroutine comes to the rescue.

PythonCopy

import math

def cash_return_coro(percent: float, years: int) -> float:
    value = math.pow(1 + percent / 100, years)
    while True:
        try:
            deposit = (yield)
            yield round(deposit * value, 2)
        except GeneratorExit:
            print('Выход из корутины')
            raise

Let's run the coroutine with the same conditions — 5 years and 5% per year.

PythonCopy

coro = cash_return_coro(5, 5)
next(coro)
values = [1000, 2000, 5000, 10000, 100000]
for item in values:
    print(coro.send(item))
    next(coro)
coro.close()

We get:

PythonCopy

1276.28
2552.56
6381.41
12762.82
127628.16
Выход из корутины

In the cash_return_coro function there's no need to compute the value variable every time you want to calculate the sum. A downside of this approach is the larger amount of code required to make it work correctly.

Using coroutines

Let's write a small coroutine implementation that can store the history of a function's results and print them.

PythonCopy

import operator

def func_manager():
    history = []
    while True:
        x, y, func = (yield)
        if func == "h":
            print(history)
            continue
        result = func(x, y)
        print(result)
        history.append(result)

manager = func_manager()
print(type(manager))

manager.send(None)
manager.send((1, 2, operator.add))
manager.send((100, 20, operator.sub))
manager.send((5, 15, operator.mul))
manager.send((None, None, "h"))
manager.close()

PythonCopy

>> output
<class 'generator'>
3
80
75
[3, 80, 75]

We created a generator and primed it with manager.send(None), then feed in the input data and store the result of each call in a variable.

Now let's simplify the program and get rid of the need to prime the generator manually. To solve this problem we'll use a decorator.

PythonCopy

import operator

def coroutine(f):
    def wrap(*args,**kwargs):
        gen = f(*args,**kwargs)
        gen.send(None)
        return gen
    return wrap

@coroutine
def func_manager():
    history = []
    while True:
        x, y, func = (yield)
        if func == "h":
            print(history)
            continue
        result = func(x, y)
        print(result)
        history.append(result)

manager = func_manager()
print(type(manager))

manager.send((1, 2, operator.add))
manager.send((100, 20, operator.sub))
manager.send((5, 15, operator.mul))
manager.send((None, None, "h"))
manager.close()

We've ended up with an example of how you can write your own coroutines.

Advantages of using coroutines

Efficiency. A multitasking program uses resources more efficiently. The main code isn't blocked to let a helper module run. Instead, they work asynchronously.

Convenience for the user. Coroutines switch quickly, so to the user it looks like tasks are running simultaneously. They don't have to wait long for the program to respond. They keep working with it while the asynchronous coroutines quietly perform additional actions.

Reduced system load. Asynchrony makes it possible to perform several actions within a single thread instead of multiplying threads. It's easier for the system to run one thread than several.

Control flexibility. Switching between coroutines happens manually. The programmer specifies this moment in the code, so control over coroutines can be managed. There's no risk that the program will switch between blocks of code at an inappropriate moment, as happens with less flexible solutions.

Drawbacks of coroutines

Complexity and a high entry barrier. Understanding how coroutines work and learning to write asynchronous code can be difficult. That's why specialists start studying coroutines only after they've already got a good grasp of the basic principles of their chosen language.

Narrow specialization. Coroutines are a high-level solution. You won't be able to speed up complex computations with coroutines. In that case you need multithreading, not asynchrony. That said, coroutines are well suited to reducing system load.

What to remember about coroutines

Advantages of coroutines in Python:

A coroutine is a mechanism for asynchronous interaction, controlled programmatically rather than by the system, unlike threads or processes.

A coroutine is less resource-intensive than threads.

A coroutine is more flexible to use than threads.

A coroutine doesn't speed up a program, but helps make its work more optimal.

Drawbacks of coroutines in Python:

A high entry barrier to understanding the topic.

An increased level of involvement in the program's operation and its further maintenance.