An iterable in Python is any object capable of returning its members one at a time, allowing it to be looped over in a for loop or processed by functions that expect a sequence of data. At its core, an iterable implements the __iter__ method, which returns an iterator object, or the __getitem__ method with sequential integer indexes starting from zero. This fundamental concept powers much of Python’s syntax, from simple list comprehensions to complex data processing pipelines. Understanding the distinction between an iterable and an iterator is the key to mastering Python’s lazy evaluation model and writing memory-efficient code Nothing fancy..
The Protocol Behind Iteration
Python relies on a specific protocol to make iteration work easily across built-in types and custom classes. When you write for item in my_object:, the interpreter performs a series of well-defined steps behind the scenes Easy to understand, harder to ignore..
First, it calls iter(my_object). iter(). If that method exists, it must return an **iterator** object. Now, this built-in function attempts to invoke my_object. getitem(0), my_object.If __iter__ is absent, Python falls back to the sequence protocol, calling my_object.__getitem__(1), and so on, until an IndexError is raised.
This is where a lot of people lose the thread It's one of those things that adds up..
The object returned by __iter__ (or created implicitly via __getitem__) is an iterator. An iterator is a stateful object that implements the __next__ method. Also, every time the loop needs the next value, it calls next(iterator), which executes __next__(). When the data is exhausted, __next__ raises a StopIteration exception, signaling the loop to terminate cleanly.
This separation—iterable (the container or source) vs. iterator (the cursor)—is crucial. An iterable can produce multiple independent iterators (like a book you can bookmark in different places), whereas an iterator is a single-use cursor (like a bookmark that only moves forward) It's one of those things that adds up..
Not the most exciting part, but easily the most useful Easy to understand, harder to ignore..
Common Built-in Iterables
Python’s standard library is rich with iterable types. Recognizing them helps you choose the right tool for the job Turns out it matters..
- Sequences:
list,tuple,str,bytes,bytearray,range. These support__getitem__andlen(), allowing random access. - Collections:
set,frozenset,dict. Sets iterate over unique elements; dictionaries iterate over keys by default (use.values()or.items()for other views). - File Objects: Open file handles are iterable, yielding lines one by one. This enables memory-efficient processing of massive files without loading them entirely into RAM.
- Generators: The result of a generator function (using
yield) or a generator expression(x for x in data). These are iterators and iterables simultaneously. - Itertools and Views: Objects like
map,filter,zip,enumerate,reversed, and dictionary view objects (dict_keys,dict_values) are lazy iterables that compute values on demand.
Iterables vs. Iterators: A Practical Distinction
The most common point of confusion for developers is the difference between the two. Consider this snippet:
my_list = [1, 2, 3] # This is an Iterable
my_iter = iter(my_list) # This is an Iterator
print(next(my_iter)) # Output: 1
print(next(my_iter)) # Output: 2
my_list is the iterable. You can call iter(my_list) as many times as you want, getting a fresh iterator each time. my_iter is the iterator. It maintains internal state (the current index). Also, once exhausted, it is useless; calling next() again raises StopIteration. You cannot "reset" an iterator.
Why does this matter?
If you pass an iterator to a function that consumes it (like list() or sum()), it becomes empty for any subsequent use. An iterable, however, can be passed around safely because a new iterator is created every time iteration starts.
def consume_twice(data):
print("First pass:", list(data))
print("Second pass:", list(data)) # Empty if data was an iterator!
consume_twice([1, 2, 3]) # Works: list is iterable
consume_twice(iter([1, 2, 3])) # Fails silently on second pass: iterator exhausted
Creating Custom Iterables
Building your own iterable classes allows you to represent complex data streams—database cursors, API pagination, or mathematical sequences—as native Python objects.
Approach 1: The Iterator Class (Explicit Protocol)
You define a class with __iter__ returning self and a __next__ method managing state That alone is useful..
class Countdown:
def __init__(self, start):
self.current = start
def __iter__(self):
return self # The instance is its own iterator
def __next__(self):
if self.current <= 0:
raise StopIteration
val = self.current
self.
for num in Countdown(3):
print(num) # 3, 2, 1
Limitation: Because __iter__ returns self, you can only have one active iteration at a time. Nested loops over the same instance will interfere with each other Took long enough..
Approach 2: The Iterable Class (Separate Iterator)
Best practice dictates separating the iterable (the data source) from the iterator (the cursor).
class CountdownIterable:
def __init__(self, start):
self.start = start
def __iter__(self):
# Return a *new* iterator instance every time
return CountdownIterator(self.start)
class CountdownIterator:
def __init__(self, start):
self.current = start
def __iter__(self):
return self
def __next__(self):
if self.Worth adding: current <= 0:
raise StopIteration
val = self. current
self.
cd = CountdownIterable(3)
print(list(cd)) # [3, 2, 1]
print(list(cd)) # [3, 2, 1] - Works perfectly because a fresh iterator is created
Approach 3: Generator Functions (The Pythonic Way)
For most use cases, writing a class is overkill. A generator function using yield automatically creates an iterator object with __iter__ and __next__ implemented correctly.
def countdown_gen(start):
while start > 0:
yield start
start -= 1
# This returns a generator iterator, which is also an iterable
for n in countdown_gen(3):
print(n)
Generators are lazy, memory-efficient, and readable. They should be your default choice unless you need complex state management or multiple methods on the iterator object.
Lazy vs. Eager Evaluation
Iterables are the gateway to lazy evaluation. An eager operation (like list(), sum(), max(), or a list comprehension [x for x in data]) consumes the entire iterable immediately, allocating memory for all results.
A lazy iterable (like range, map, filter, or a generator) produces items only when requested. This distinction is vital for performance.
# Eager: Creates a list of 1,000,000 integers in memory immediately
eager_list = [x * 2 for x in range(1_000_000)]
# Lazy: Creates a generator object (tiny memory footprint)
lazy_gen = (x * 2 for x in range(1_000_000))
# Processing the first 5 items:
print
To limit it to the first five values, use `itertools.islice`:
```python
from itertools import islice
first_five = islice(lazy_gen, 5)
print(list(first_five)) # [0, 2, 4, 6, 8]
The original generator is now partially consumed, but you can create fresh generators whenever you need a new sequence.
For simple numeric ranges, Python already provides a lazy, immutable option:
from itertools import islice
print(list(islice(range(1_000_000), 5))) # [0, 1, 2, 3, 4]
This avoids building a million-element list just to read a few numbers.
Lazy evaluation becomes especially powerful when working with infinite sequences or large streams of data. Instead of materializing everything at once, you process items only as they are requested, which keeps memory usage low and pipelines clean.
Conclusion
Python’s iteration protocol—__iter__ and __next__—is simple but flexible. The core design choice is whether an iterable creates a new iterator on each call to iter() or returns itself. The former supports independent, nested iterations; the latter allows only one active iterator at a time.
Generators offer the most Pythonic solution for everyday iteration, automatically handling iterator creation and state without boilerplate. Custom iterator classes remain valuable when you need explicit control over complex behavior And that's really what it comes down to..
Lazy evaluation is a key benefit of iterators and generators. It lets you process data incrementally, conserve memory, and build efficient data pipelines. Whether you use range, generators, or custom classes, understanding this distinction helps you write faster, cleaner Python Still holds up..