Iterators in Python Explained: Understanding __next__(), StopIteration, and One-Way Iteration

Hey Python developers! 👋

Welcome back to our journey through Python's iteration system.

In Part 1, we explored iterables using a buffet table metaphor.

The buffet contained all the food.

But here's the thing:

You don't eat directly from the buffet.

Someone has to serve you.

That someone is the iterator.

Think of an iterator as a waiter carrying dishes from the buffet to your table.

The waiter remembers:

Where they currently are

but forgets everything behind them.

They move in one direction only.

Forward.

Never backward.

In this article, we'll explore:

• What iterators really are
• How __iter__() and __next__() work
• What happens inside memory
• Why iterators can't rewind
• Common iterator mistakes
• Building custom iterators
• Useful iterator tricks
• Performance considerations

Let's meet Python's hardest-working waiters. 🍴🤖

Buffet vs Waiter

Let's quickly revisit the metaphor.

Iterable = Buffet Table

Contains all available items.

Example:

numbers = [1, 2, 3]

The buffet remains available.

You can revisit it repeatedly.

Iterator = Waiter

Tracks progress.

Example:

it = iter(numbers)

The waiter knows:

Current position

and serves items one at a time.

Every iterator is temporary.

Once it reaches the end:

Job finished.

What Makes an Iterator?

In Python, an iterator is any object that implements:

__iter__()

Returns the iterator itself.

__next__()

Returns the next item.

Or raises:

StopIteration

when exhausted.

This is the complete iterator protocol.

A Simple Example

nums = [1, 2, 3]

it = iter(nums)

Retrieve values:

print(next(it))

Output:

1

Again:

print(next(it))

Output:

2

Again:

print(next(it))

Output:

3

One more time:

print(next(it))

Output:

StopIteration

The waiter has no more dishes to serve.

What Does a for Loop Actually Do?

Many developers think:

for item in numbers:
    print(item)

is special syntax.

Internally it's approximately:

it = iter(numbers)

while True:
    try:
        item = next(it)
        print(item)
    except StopIteration:
        break

Every for loop in Python relies on iterators.

Always.

Understanding StopIteration

This exception is not an error.

It's a signal.

The iterator says:

"I'm finished. Nothing left."

Example:

class EmptyIterator:
    def __iter__(self):
        return self

    def __next__(self):
        raise StopIteration

The first call to:

next(obj)

immediately ends iteration.

Python uses this mechanism everywhere.

What's Stored Inside an Iterator?

Many beginners imagine iterators copy data.

They don't.

Consider:

numbers = [10, 20, 30]

Create:

it = iter(numbers)

Typically, the iterator stores:

Reference to original list
+
Current index

That's it.

No data duplication.

No extra list creation.

Why Iterators Are Memory Efficient

Imagine:

numbers = list(range(10_000_000))

The list consumes significant memory.

But:

it = iter(numbers)

adds almost nothing.

The iterator only remembers:

Current position

This makes iterators ideal for:

• Streaming files
• Processing large datasets
• Data pipelines
• Network responses

Why Iterators Can't Rewind

A common question:

Why can't I restart an iterator?

Because iterators don't maintain history.

Imagine a waiter delivering food.

The waiter remembers:

Current table

but not every table already visited.

Similarly:

it = iter([1, 2, 3])

tracks only:

Current position

Once the end is reached:

Finished.

No rewind button exists.

The Classic Iterator Mistake

Consider:

it = iter([1, 2, 3, 4])

print(sum(it))

Output:

10

Now:

print(list(it))

Output:

[]

Why?

Because:

sum()

already consumed every value.

The waiter already served all dishes.

Nothing remains.

How to Fix Exhausted Iterators

Use the iterable instead.

Example:

numbers = [1, 2, 3, 4]

print(sum(numbers))
print(list(numbers))

Output:

10
[1, 2, 3, 4]

The iterable can create fresh iterators whenever needed.

Building Your Own Iterator

Let's create a custom iterator.

class Counter:
    def __init__(self, limit):
        self.limit = limit
        self.current = 0

    def __iter__(self):
        return self

    def __next__(self):
        if self.current >= self.limit:
            raise StopIteration

        self.current += 1
        return self.current

Usage:

for n in Counter(3):
    print(n)

Output:

1
2
3

Our waiter serves three values and retires.

Understanding the Flow

Iteration proceeds like this:

Counter
   |
   V
__iter__()
   |
   V
Iterator
   |
   V
__next__()
   |
   V
1
2
3
StopIteration

This is exactly how Python's built-in iterators work.

Iterator Objects Are Usually Their Own Iterators

Notice:

def __iter__(self):
    return self

Why?

Because iterator objects already know their own state.

They don't need another iterator.

This differs from iterables like lists.

Example:

numbers = [1, 2, 3]

Each call to:

iter(numbers)

creates a new iterator.

The iter(callable, sentinel) Trick

One of Python's lesser-known features.

Example:

import random

dice = iter(
    lambda: random.randint(1, 6),
    6
)

Now:

for roll in dice:
    print(roll)

The callable executes repeatedly.

Iteration stops when:

6

is returned.

Think:

Keep rolling until six appears.

Very elegant.

Infinite Iterators

Some iterators never stop.

Example:

from itertools import count

counter = count()

Produces:

0
1
2
3
...

forever.

Use carefully.

Infinite loops happen surprisingly easily.

Controlling Infinite Iterators

Example:

from itertools import count, islice

for value in islice(count(), 5):
    print(value)

Output:

0
1
2
3
4

islice() acts like a safety fence.

Iterator Pipelines

One of Python's greatest strengths.

Example:

numbers = range(10)

Filter:

evens = filter(
    lambda x: x % 2 == 0,
    numbers
)

Transform:

squares = map(
    lambda x: x * x,
    evens
)

Consume:

print(list(squares))

Output:

[0, 4, 16, 36, 64]

Each stage processes values lazily.

No giant intermediate lists required.

Lists vs Iterators vs Generators

Choosing the right tool matters.

When Should You Use Each?

Use Lists When

• Multiple passes are needed
• Random access is important
• Data fits comfortably in memory

Use Iterators When

• Processing streams
• Reading files
• Building pipelines
• Working with huge datasets

Use Generators When

• Creating custom lazy sequences
• Simplifying iterator implementation

We'll dive deeply into generators in Part 3.

A Visual Mental Model

Iterable (Buffet Table)
           |
           |
           | iter()
           V
Iterator (Waiter)
           |
           |
           | next()
           V
      Next Item
           |
           V
    StopIteration

Simple.

Powerful.

Pythonic.

Common Iterator Pitfalls

Sharing One Iterator

Bad:

it = iter(data)

consumer1(it)
consumer2(it)

The second consumer may receive partial data.

Assuming Reusability

Bad:

it = iter(data)

list(it)
list(it)

Second result:

[]

Forgetting Stop Conditions

Infinite iterators without safeguards can lock programs indefinitely.

Always think about termination.

TL;DR Quick Recap

• Iterators implement __iter__() and __next__().
• next() retrieves one item at a time.
• StopIteration signals completion.
• Every for loop uses iterators internally.
• Iterators store minimal state.
• They are memory efficient because they don't copy data.
• Iterators move forward only.
• Exhausted iterators cannot be reused.
• Custom iterators are straightforward to implement.
• Lazy pipelines make iterators extremely powerful.

Final Thoughts: Iterators Power Almost Everything 🧠

Most Python developers use iterators every day without realizing it.

Whenever you write:

for item in something:
    ...

an iterator is quietly working behind the scenes.

It tracks progress.

Delivers values.

Signals completion.

Consumes almost no extra memory.

Understanding iterators unlocks deeper knowledge of:

• Generators
• Comprehensions
• File streaming
• Data pipelines
• Async programming
• Python internals

And speaking of generators...

that's where we're headed next.

A Little Python Joke to End On 😄

Why did the iterator quit its restaurant job?

Because after serving every dish once, it couldn't remember where it started.

Frequently Asked Questions

What is an iterator in Python?

An iterator is an object that returns values one at a time using:

__next__()

until:

StopIteration

is raised.

What is the difference between an iterable and an iterator?

An iterable can produce iterators.

An iterator tracks progress and produces values.

Think:

Iterable → Buffet
Iterator → Waiter

What does next() do?

It requests the next available value from an iterator.

Example:

next(iterator)

What is StopIteration?

A special exception that signals the iterator has no more items.

Python uses it internally to stop loops.

Why are iterators memory efficient?

Because they usually store only:

• A reference to data
• Current position

rather than duplicating all values.

Why can't iterators be reused?

Because they move forward only.

Once exhausted, they remain finished.

What is iter(callable, sentinel)?

A special form of iter() that repeatedly calls a function until a sentinel value is returned.

Example:

iter(func, stop_value)

What are infinite iterators?

Iterators that never naturally terminate.

Example:

itertools.count()

Are generators iterators?

Yes.

Generators automatically implement the iterator protocol.

We'll explore them deeply in Part 3.

When should I use iterators?

Use iterators when:

• Processing large datasets
• Streaming files
• Building lazy pipelines
• Conserving memory

Key Takeaways

• Iterators are the engines behind Python's iteration system.
• They implement __iter__() and __next__().
• StopIteration signals completion.
• Iterators consume minimal memory.
• They move forward only and cannot rewind.
• Custom iterators are easy to build.
• Iterator pipelines enable scalable data processing.
• Understanding iterators makes generators and advanced Python concepts much easier to learn.

Series Navigation

Part 1: Iterables — The Buffet Table 🍽️

Part 2: Iterators — The Waiters with One-Way Memory 🍴🤖 (Current Article)

Part 3: Generators — The Lazy Magicians of Python 🎩🐍

If you found this helpful, share it with another Python developer and follow for more Python deep dives, language internals, software architecture discussions, and backend engineering insights. 🚀

About the Author

Anik Sikder is a Software Engineer specializing in Backend Development, Python, Django, FastAPI, DevOps, Cloud Infrastructure, and Software Architecture.

He writes about Python internals, distributed systems, scalable backend platforms, system design, and modern software engineering practices.