24.generators

A generator is a function that returns an object (iterator) which we can iterate over (one value at a time).

Generator functions allow us to write a function that can send back a value and then later resume to pick up where it left off.

Create Generators

It is fairly simple to create a generator in Python. It is as easy as defining a normal function, but with a yield statement instead of a return statement.

If a function contains at least one yield statement (it may contain other yield or return statements), it becomes a generator function. Both yield and return will return some value from a function.

The difference is that while a return statement terminates a function entirely, yield statement pauses the function saving all its states and later continues from there on successive calls.

Example

# A simple generator function
def my_gen():
    n = 1
    print('This is printed first')
    # Generator function contains yield statements
    yield n

    n += 1
    print('This is printed second')
    yield n

    n += 1
    print('This is printed at last')
    yield n

An interactive run in the interpreter is given below

>>> # It returns an object but does not start execution immediately.
>>> a = my_gen()

>>> # We can iterate through the items using next().
>>> next(a)
This is printed first
1
>>> # Once the function yields, the function is paused and the control is transferred to the caller.

>>> # Local variables and theirs states are remembered between successive calls.
>>> next(a)
This is printed second
2

>>> next(a)
This is printed at last
3

>>> # Finally, when the function terminates, StopIteration is raised automatically on further calls.
>>> next(a)
Traceback (most recent call last):
...
StopIteration
>>> next(a)
Traceback (most recent call last):
...
StopIteration

We can use generators with for loops directly.

This is because a for loop takes an iterator and iterates over it using next() function. It automatically ends when StopIteration is raised.

Example

# A simple generator function
def my_gen():
    n = 1
    print('This is printed first')
    # Generator function contains yield statements
    yield n

    n += 1
    print('This is printed second')
    yield n

    n += 1
    print('This is printed at last')
    yield n


# Using for loop
for item in my_gen():
    print(item)

Output:
This is printed first
1
This is printed second
2
This is printed at last
3

Python Generators with a loop

Generators can be utilised better when we implement them with a loop with a suitable terminating condition.

Fibonacci Sequence

def gen_fibon(n):
    a = 1
    b = 1
    for i in range(n):
        yield(a)
        a,b = b, a+b

for number in gen_fibon(10):
print(number)

list(gen_fibon(10))

Output:
1
1
2
3
5
8
13
21
34
55

[1, 1, 2, 3, 5, 8, 13, 21, 34, 55]

We can call the above in below manner to get the result one by one

g = gen_fibon(10)

print(next(g))
print(next(g))
print(next(g))
print(next(g))
print(next(g))
print(next(g))

Output:
1
1
2
3
5
8

Generator Expressions

Simple generators can be easily created on the fly using generator expressions. It makes building generators easy.

Similar to the lambda functions which create anonymous functions, generator expressions create anonymous generator functions.

The syntax for generator expression is similar to that of a list comprehension in Python. But the square brackets are replaced with round parentheses.

The major difference between a list comprehension and a generator expression is that a list comprehension produces the entire list while the generator expression produces one item at a time.

Example

# Initialize the list
my_list = [1, 3, 6, 10]

# square each term using list comprehension
list_ = [x**2 for x in my_list]

# same thing can be done using a generator expression
# generator expressions are surrounded by parenthesis ()
generator = (x**2 for x in my_list)

print(list_)
print(generator)

Output

[1, 9, 36, 100]
<generator object <genexpr> at 0x7f5d4eb4bf50>

In the above example, generator expression did not produce the required result immediately. Instead, it returned a generator object, which produces items only on demand.

To start getting items from the generator.

Example

# Initialize the list
my_list = [1, 3, 6, 10]

a = (x**2 for x in my_list)
print(next(a))

print(next(a))

print(next(a))

print(next(a))

next(a)

Output

1
9
36
100
Traceback (most recent call last):
  File "<string>", line 15, in <module>
StopIteration

Generator expressions can be used as function arguments.

Example

>>> sum(x**2 for x in my_list)
146

>>> max(x**2 for x in my_list)
100