Python Iterators and Generators
Table of Contents
Description
Iterators are objects that allow sequential access to elements using the __iter__() and __next__() methods. Generators are a simpler way to create iterators using yield instead of return. Generators are memory efficient and lazy, meaning they generate items one at a time and only when required.
Prerequisites
- Python loops (for, while)
- Understanding of functions
- Knowledge of classes (for custom iterators)
- Exception handling (StopIteration)
Examples
Here's a simple program in Python:
✅ Iterator using __iter__() and __next__()
class Counter:
def __init__(self, start, end):
self.current = start
self.end = end
def __iter__(self):
# Returns the iterator object itself
return self
def __next__(self):
# Returns the next value or raises StopIteration
if self.current <= self.end:
num = self.current
self.current += 1
return num
else:
raise StopIteration
# Using the iterator
c = Counter(1, 5)
for num in c:
print(num) # Output: 1 2 3 4 5
✅ Generator Function using yield
def countdown(n):
while n > 0:
yield n # Pause and resume here
n -= 1
# Using the generator
for i in countdown(5):
print(i) # Output: 5 4 3 2 1
✅ Generator Expression
# Similar to list comprehension but lazy
squares = (x*x for x in range(5))
for num in squares:
print(num) # Output: 0 1 4 9 16
✅ Generator vs List (Memory Comparison)
import sys
lst = [x for x in range(10000)]
gen = (x for x in range(10000))
print(sys.getsizeof(lst)) # Larger memory
print(sys.getsizeof(gen)) # Much smaller memory
Real-World Applications
File reading line by line (yield)
Streaming large datasets (e.g., CSV, logs)
Data pipelines (ETL systems)
Implementing infinite sequences (e.g., Fibonacci)
Memory optimization in ML and web scraping
Where topic Can Be Applied
Backend APIs for pagination and streaming
Machine learning model input pipelines (e.g., TensorFlow datasets)
Asynchronous programming and coroutines
Lazy evaluation and on-the-fly data generation
Custom collection classes
Resources
Topic video source
A comprehensive video
Watch
Python pdf
pdf on topic
Visit
Interview Questions
What is the difference between an iterable and an iterator?
What are the advantages of using generators?
How does yield differ from return?
Can we iterate multiple times over a generator?
What does StopIteration do?
How can we create an infinite generator?
Difference between generator expressions and list comprehensions?
How do __iter__() and __next__() work in a class?
How does memory usage differ between a generator and a list?
Can a generator be reused once it is exhausted?