Python Iterators and Generators
Table of Contents
Description
Iterators are objects that allow sequential access to elements using the __iter__() and __next__() methods. Generators are a simpler way to create iterators using yield instead of return. Generators are memory efficient and lazy, meaning they generate items one at a time and only when required.
Prerequisites
- Python loops (for, while)
- Understanding of functions
- Knowledge of classes (for custom iterators)
- Exception handling (StopIteration)
Examples
Here's a simple program in Python:
✅ Iterator using __iter__() and __next__() class Counter: def __init__(self, start, end): self.current = start self.end = end def __iter__(self): # Returns the iterator object itself return self def __next__(self): # Returns the next value or raises StopIteration if self.current <= self.end: num = self.current self.current += 1 return num else: raise StopIteration # Using the iterator c = Counter(1, 5) for num in c: print(num) # Output: 1 2 3 4 5 ✅ Generator Function using yield def countdown(n): while n > 0: yield n # Pause and resume here n -= 1 # Using the generator for i in countdown(5): print(i) # Output: 5 4 3 2 1 ✅ Generator Expression # Similar to list comprehension but lazy squares = (x*x for x in range(5)) for num in squares: print(num) # Output: 0 1 4 9 16 ✅ Generator vs List (Memory Comparison) import sys lst = [x for x in range(10000)] gen = (x for x in range(10000)) print(sys.getsizeof(lst)) # Larger memory print(sys.getsizeof(gen)) # Much smaller memoryReal-World Applications
File reading line by line (yield)
Streaming large datasets (e.g., CSV, logs)
Data pipelines (ETL systems)
Implementing infinite sequences (e.g., Fibonacci)
Memory optimization in ML and web scraping
Where topic Can Be Applied
Backend APIs for pagination and streaming
Machine learning model input pipelines (e.g., TensorFlow datasets)
Asynchronous programming and coroutines
Lazy evaluation and on-the-fly data generation
Custom collection classes
Resources
WatchTopic video source
A comprehensive video
VisitPython pdf
pdf on topic
Interview Questions
What is the difference between an iterable and an iterator?
What are the advantages of using generators?
How does yield differ from return?
Can we iterate multiple times over a generator?
What does StopIteration do?
How can we create an infinite generator?
Difference between generator expressions and list comprehensions?
How do __iter__() and __next__() work in a class?
How does memory usage differ between a generator and a list?
Can a generator be reused once it is exhausted?