Last modified: Nov 19, 2026

Python Generators and yield Explained

Python generators are one of the most useful features in the language. They let you produce values one at a time instead of all at once. This saves memory and keeps your code clean.

The yield keyword is the heart of every generator. In this guide, you will learn what generators are, how yield works, and when to use them. We will also cover common mistakes and real examples.

What Is a Generator?

A generator is a special kind of function. It returns an iterator that produces values lazily. That means it gives you one value, pauses, and resumes later.

Regular functions run from start to finish. They return a single value and exit. Generators can pause in the middle and continue where they left off.

You create a generator by using yield instead of return. The function object you get back is a generator object.

Your First Generator Example

Let's start with a simple generator. It produces the numbers 1, 2, and 3.


def count_up():
    yield 1          # pause here and send 1
    yield 2          # resume later and send 2
    yield 3          # resume again and send 3

gen = count_up()     # create the generator object
print(next(gen))     # get the first value
print(next(gen))     # get the second value
print(next(gen))     # get the third value

1
2
3

Notice that nothing runs when you call count_up(). The function body only executes when you ask for a value with next().

Each call to next runs the code until the next yield. Then it pauses and remembers its state.

How yield Works Under the Hood

When Python hits a yield statement, it does three things. It returns the value to the caller. It saves the function's local state. It pauses execution.

When you call next() again, Python resumes right after the yield. Local variables keep their values. This is why generators can remember where they were.

When the function ends, the generator raises StopIteration. A for loop handles this automatically for you.


def count_up():
    yield 1
    yield 2

for value in count_up():   # for loop stops at StopIteration
    print(value)

1
2

Generators vs Lists

A list stores every value in memory. A generator produces values on demand. For small data, the difference is minor. For large data, it matters a lot.

Imagine you need one million numbers. A list holds all of them at once. A generator holds just one at a time.


# List: builds the whole thing in memory
numbers_list = [x * x for x in range(1000000)]

# Generator: computes values one by one
numbers_gen = (x * x for x in range(1000000))

print(type(numbers_list))  # list
print(type(numbers_gen))   # generator

This lazy behavior is called lazy evaluation. It is the main reason generators are memory-friendly.

Generator Expressions

You can create a generator in one line. Use parentheses instead of square brackets. This is called a generator expression.

It looks like a list comprehension but produces values lazily. This is great when you only need to loop once.


squares = (n * n for n in range(5))

for s in squares:
    print(s)

0
1
4
9
16

Generator expressions pair nicely with built-in functions. If you have used the Python map function, you already know it returns an iterator. Generators work the same way.

Using Generators with for Loops

Most of the time, you will use a generator inside a for loop. The loop calls next until the generator is empty.

This pattern is clean and readable. You never call next() yourself. Python handles the stopping condition for you.


def evens(limit):
    n = 0
    while n < limit:
        yield n       # send the current even number
        n += 2        # prepare the next one

for num in evens(10):
    print(num)

0
2
4
6
8

Infinite Generators

Generators can run forever. Since values are produced on demand, an infinite generator is safe. You just stop consuming it when you are done.

A list can never be infinite. A generator can. This is a powerful difference.


def natural_numbers():
    n = 1
    while True:        # infinite loop
        yield n
        n += 1

gen = natural_numbers()

for _ in range(5):     # take only the first 5 values
    print(next(gen))

1
2
3
4
5

This trick is common in data pipelines. You read chunks until some condition is met. The generator never needs to know the total size.

yield from: Delegating to Another Generator

Sometimes a generator needs to yield values from another iterable. The yield from statement makes this easy.

It delegates iteration to a sub-generator. This keeps your code flat and readable.


def first_half():
    yield 1
    yield 2

def second_half():
    yield 3
    yield 4

def combined():
    yield from first_half()   # delegate to the first generator
    yield from second_half()  # then delegate to the second

print(list(combined()))

[1, 2, 3, 4]

Sending Values Into a Generator

Generators can also receive values. The send method pushes a value into the generator. The yield expression then returns that value.

This turns a generator into a simple coroutine. You can build stateful pipelines without classes.


def echo():
    while True:
        received = yield   # yield returns the sent value
        print("Got:", received)

gen = echo()
next(gen)          # start the generator
gen.send("hello")  # send a value in
gen.send("world")

Got: hello
Got: world

You must call next once before sending. This advances the generator to the first yield.

Common Mistakes

Beginners often hit a few predictable errors. Knowing them early saves time.

Mistake 1: Trying to reuse a generator. Generators are exhausted after one pass. Loop over it again and you get nothing.


gen = (x for x in range(3))

print(list(gen))   # first pass works
print(list(gen))   # second pass is empty

[0, 1, 2]
[]

Mistake 2: Forgetting to call the generator function. Calling gen() gives you the generator object. Calling gen without parentheses gives you the function itself.

Mistake 3: Mixing return and yield carelessly. A return inside a generator ends it early. The returned value appears in StopIteration.

Real Use Cases

Generators shine in many everyday tasks. Here are common patterns you will meet.

Reading large files: Process one line at a time instead of loading the whole file. Memory stays flat no matter the file size.


def read_lines(path):
    with open(path) as f:
        for line in f:
            yield line.strip()   # one line at a time

# Usage: for line in read_lines("big.txt"): ...

Data pipelines: Chain generators to filter, transform, and combine data. Each stage is lazy and small.

Generators also pair well with tools like Python's filter function. Both return iterators, so they compose cleanly.

Fibonacci sequences: Generate numbers on demand. No need to store a huge list.


def fibonacci():
    a, b = 0, 1
    while True:
        yield a
        a, b = b, a + b

gen = fibonacci()
for _ in range(8):
    print(next(gen))

0
1
1
2
3
5
8
13

This is a classic interview question. The generator version is short and memory-safe.

Generators and Other Python Tools

Generators fit naturally with many built-in functions. Tools like Python's zip function combine iterables and return iterators. You can chain them with generators for compact code.

If you are still learning functions, review how *args and **kwargs work in Python. Generators often accept flexible arguments in real projects.

Remember that generators are iterators. They work wherever Python expects an iterable. That includes loops, comprehensions, and many standard library functions.

When Should You Use Generators?

Use a generator when you process data one item at a time. Use it when the data is large or infinite. Use it when you want lazy evaluation.

Use a list when you need random access. Use a list when you need to loop multiple times. Use a list when the data is small and speed matters.

The rule is simple: stream it with a generator, store it with a list. Pick based on your needs.

Conclusion

Python generators and the yield keyword make lazy iteration easy. A generator pauses at each yield and resumes on demand. This saves memory and keeps code simple.

Start with small examples. Write a generator for a range of numbers. Then try infinite sequences and file readers. Practice is the fastest way to master them.

Once generators click, you will reach for them often. They are a core Python skill worth learning well.