Iterables in Python Explained: Understanding the Iteration Protocol with a Buffet Table Metaphor

Hey Python developers! 👋

When you write:

for x in something:
    print(x)

everything feels simple.

Python loops through values.

You get your output.

Life is good.

But have you ever wondered:

What actually makes an object loopable?

Why can you iterate over:

[1, 2, 3]

but also over:

"hello"

or:

{"a": 1, "b": 2}

or even:

open("data.txt")

The answer lies in one of Python's most elegant design patterns:

The Iteration Protocol

Understanding iterables is the foundation for mastering:

• Loops
• Generators
• Comprehensions
• File processing
• Data pipelines
• Lazy evaluation

In this article (Part 1 of a 3-part series), we'll explore:

• What an iterable actually is
• How iter() works internally
• Memory and performance implications
• Real-world applications
• Designing reusable custom iterables

And to make everything memorable, we'll use a buffet restaurant metaphor. 🍽️

Let's dive in.

The Buffet Table Metaphor 🍴

Imagine you're at a buffet restaurant.

The restaurant contains:

🍕 🍔 🌮 🍝 🍰

All arranged neatly on a buffet table.

In our analogy:

Buffet Table → Iterable
Plate         → Iterator

The buffet contains all available food.

The plate keeps track of what you've already taken.

Why This Analogy Works

The buffet itself never gets consumed.

You can always grab a fresh plate and start again.

Example:

foods = ["Pizza", "Burger", "Taco"]

You can loop multiple times:

for food in foods:
    ...

and again:

for food in foods:
    ...

The iterable remains unchanged.

Each loop receives a fresh iterator.

Multiple People, Multiple Plates

Imagine two people eating from the same buffet.

Each has a different plate.

Person A → Plate A
Person B → Plate B

They track progress independently.

Similarly:

numbers = [1, 2, 3]

it1 = iter(numbers)
it2 = iter(numbers)

Each iterator maintains its own position.

This is one of the most important concepts in Python iteration.

What Is an Iterable?

An iterable is any object that can produce an iterator.

More formally:

An object is iterable if it:

Option 1: Implements __iter__()

obj.__iter__()

returns an iterator.

Option 2: Implements __getitem__()

Using:

obj[0]
obj[1]
obj[2]
...

Python can simulate iteration until:

IndexError

occurs.

This older mechanism still works today.

Meet iter(): The Universal Adapter

Whenever Python needs to iterate, it calls:

iter(obj)

Think of iter() as a universal adapter.

Internally, Python follows roughly this process:

1. Does object have __iter__()?
   → Use it

2. Otherwise, does it support __getitem__()?
   → Use index-based iteration

3. Otherwise
   → TypeError

If neither protocol exists:

iter(obj)

fails.

Common Built-In Iterables

Python provides many iterable types.

Lists

numbers = [1, 2, 3]

Strings

text = "hello"

Tuples

coordinates = (10, 20)

Sets

letters = {"a", "b", "c"}

Dictionaries

data = {"name": "Anik"}

Files

with open("data.txt") as file:
    ...

All of these support iteration.

Using iter() and next()

Let's see iteration manually.

numbers = [1, 2, 3]

it = iter(numbers)

Retrieve items:

print(next(it))

Output:

1

Again:

print(next(it))

Output:

2

Again:

print(next(it))

Output:

3

One more time:

print(next(it))

Output:

StopIteration

The iterator signals that it's finished.

Iterables vs Iterators

These concepts are often confused.

Iterable

Container of values.

Example:

numbers = [1, 2, 3]

Can create multiple iterators.

Iterator

Tracks progress.

Example:

it = iter(numbers)

Knows:

Current position

and provides:

next()

to move forward.

Think:

Iterable = Buffet Table
Iterator = Plate

Sets Are Iterable but Not Indexable

Many beginners assume iteration requires indexing.

Not true.

Example:

letters = {"a", "b", "c"}

This works:

for letter in letters:
    print(letter)

But:

letters[0]

raises:

TypeError

because sets do not support indexing.

Iteration and indexing are separate concepts.

Memory Efficiency of Iterators

One of the biggest benefits of iterators is memory efficiency.

Consider:

numbers = [1, 2, 3]

The list stores all elements.

A list iterator stores only:

Reference to list
+
Current position

That's it.

No data copying occurs.

Why Iterators Are Cheap

Imagine:

numbers = list(range(10_000_000))

The list consumes significant memory.

But:

it = iter(numbers)

adds only a tiny overhead.

The iterator simply remembers:

Current index

This is why iterators are considered:

O(1) additional memory

The Magic of range()

A great example:

range(10_000_000)

Python doesn't store ten million integers.

Instead:

start
stop
step

are stored.

Values are generated when needed.

This approach is called:

Lazy Evaluation

and is one reason Python can handle huge ranges efficiently.

Iterables via __getitem__

An object doesn't necessarily need __iter__().

Example:

class SquareSeq:
    def __getitem__(self, index):
        if index < 0:
            raise IndexError

        return index * index

Usage:

sq = SquareSeq()

it = iter(sq)

Now:

print(next(it))

Output:

0

Then:

print(next(it))

Output:

1

Then:

print(next(it))

Output:

4

Python internally calls:

sq[0]
sq[1]
sq[2]
...

until:

IndexError

is raised.

Why IndexError Matters

Suppose you forget:

raise IndexError

Then Python has no idea when iteration should stop.

The result:

Infinite iteration

or unexpected behavior.

Always define a termination condition.

Real-World Use Cases

Iteration powers many everyday tasks.

Processing Huge Files

Instead of loading everything:

content = file.read()

stream line-by-line:

for line in file:
    ...

Memory efficient.

Paginated APIs

Fetch data lazily:

page 1
page 2
page 3

only when needed.

Data Pipelines

Combine:

map()
filter()
zip()
itertools

without creating large intermediate lists.

Lazy Computations

Generate values on demand.

Avoid unnecessary work.

Large Numeric Ranges

range(1_000_000_000)

remains lightweight.

Common Pitfalls

Exhausted Iterators

Example:

it = iter([1, 2, 3])

Consume everything:

for x in it:
    pass

Second loop:

for x in it:
    print(x)

Produces nothing.

The iterator is exhausted.

Create a new iterator instead.

Infinite Iterables

Example:

import itertools

counter = itertools.count()

Never ends.

Use safeguards such as:

itertools.islice()

or explicit stop conditions.

Assuming Ordering

Sets:

{"a", "b", "c"}

are iterable.

But ordering should not be relied upon conceptually.

Iteration order may vary depending on implementation details.

A Bad Custom Iterable Design

Many beginners combine:

Container
+
Iterator

into the same object.

Example:

class BadCities:
    def __init__(self):
        self._cities = [
            "Paris",
            "Berlin",
            "Rome"
        ]
        self._index = 0

    def __iter__(self):
        return self

    def __next__(self):
        if self._index >= len(self._cities):
            raise StopIteration

        city = self._cities[self._index]
        self._index += 1

        return city

Problem:

for city in cities:
    ...

works once.

Second loop:

for city in cities:
    ...

returns nothing.

The iterator already consumed itself.

The Better Design

Separate the container from the iterator.

Iterator

class CityIterator:
    def __init__(self, cities):
        self._cities = cities
        self._index = 0

    def __iter__(self):
        return self

    def __next__(self):
        if self._index >= len(self._cities):
            raise StopIteration

        city = self._cities[self._index]
        self._index += 1

        return city

Iterable

class Cities:
    def __init__(self):
        self._cities = [
            "Paris",
            "Berlin",
            "Rome"
        ]

    def __iter__(self):
        return CityIterator(self._cities)

Now every loop receives a fresh iterator.

Much better.

Mental Model Diagram

[Iterable (Buffet Table)]
           |
           |
           | __iter__()
           | or
           | __getitem__()
           V
[Iterator (Plate)]
           |
           |
           | __next__()
           V
      Item by Item
           |
           V
   StopIteration

This single diagram explains most of Python's iteration system.

Quick Checklist

Before designing an iterable, ask:

Multiple Passes Needed?

Use:

Iterable

not a single iterator.

Memory Efficiency Important?

Use:

Iterator
Generator

instead of large lists.

Need to Test Iterability?

try:
    iter(obj)
except TypeError:
    ...

Need Controlled Termination?

Consider:

iter(callable, sentinel)

for custom stopping conditions.

TL;DR Quick Recap

• Iterables are objects that can produce iterators.
• iter() is Python's universal iteration adapter.
• Iterables typically implement __iter__() or __getitem__().
• Iterators track progress and return items one at a time.
• Iterators use very little memory.
• Built-in types like lists, strings, sets, dictionaries, and files are iterable.
• Iterators become exhausted after use.
• Good custom iterable design separates containers from iterators.
• Lazy iteration enables scalable, memory-efficient programs.

Final Thoughts: Iterables Are the Foundation of Python 🧠

Almost every Python developer uses iteration every day.

But far fewer understand what's happening underneath.

The iterable protocol is one of Python's most elegant abstractions.

It allows completely different objects:

• Lists
• Files
• Strings
• Database cursors
• API streams
• Generators

to behave consistently inside:

for item in something:
    ...

Once you understand iterables, concepts like generators, iterators, comprehensions, and lazy evaluation become much easier to grasp.

And that's exactly where we're heading next.

A Little Python Joke to End On 😄

Why did the iterator bring a plate to the buffet?

Because it knew the buffet was reusable, but the plate only got one trip through the line.

Frequently Asked Questions

What is an iterable in Python?

An iterable is any object that can produce an iterator.

Examples include:

list
tuple
str
set
dict
file

What does iter() do?

iter() returns an iterator from an iterable object.

Example:

numbers = [1, 2, 3]

it = iter(numbers)

What is the difference between an iterable and an iterator?

An iterable produces iterators.

An iterator tracks progress and returns values one at a time.

Think:

Iterable → Buffet Table
Iterator → Plate

Can an object be iterable without __iter__()?

Yes.

Python can fall back to:

__getitem__()

using index-based access until IndexError.

Why are iterators memory efficient?

Because they store only:

• A reference to the data
• Current position

rather than copying all values.

What happens when an iterator finishes?

It raises:

StopIteration

which tells Python to stop looping.

Why can't exhausted iterators be reused?

Because they remember their current position.

Once the end is reached, they remain finished.

Create a new iterator instead.

Are sets iterable?

Yes.

Example:

for item in {"a", "b", "c"}:
    ...

works perfectly.

However, sets do not support indexing.

Why is range() memory efficient?

Because it stores only:

start
stop
step

and calculates values lazily when needed.

Should iterable containers return themselves from __iter__()?

Usually no.

Containers should generally return a fresh iterator object so multiple loops work independently.

Key Takeaways

• Iterables provide values through the iteration protocol.
• Python uses iter() and next() behind every for loop.
• __iter__() is the primary mechanism for iteration.
• __getitem__() can serve as a fallback.
• Iterators are lightweight and memory efficient.
• Lazy iteration helps process massive datasets.
• Good iterable design separates containers from iterators.
• Understanding iterables is the first step toward mastering generators and advanced Python internals.

Series Navigation

Part 1: Iterables — The Buffet Table 🍽️ (Current Article)

Part 2: Iterators — The Waiters with One-Way Memory 🍽️

Part 3: Generators — The Magical Chefs Cooking on Demand 👨‍🍳

If you found this helpful, share it with another Python developer and follow for more Python deep dives, language internals, software architecture discussions, and backend engineering insights. 🚀

About the Author

Anik Sikder is a Software Engineer specializing in Backend Development, Python, Django, FastAPI, DevOps, Cloud Infrastructure, and Software Architecture.

He writes about Python internals, scalable backend systems, system design, distributed architecture, and modern software engineering practices.