Python Generators and Iterators

What is iterators?

Iterators are used to traverse the list. for statement can be used to iterate over collections like list, dictionary etc.

for i in [1, 2, 3, 4]:                                                          
    print i, 

Output:

$ python iterators.py
1 2 3 4

Difference print i and print i,

print i : prints element in new line.
print i, : prints element in same line with space.

Iterate over a string

or ch in "python":                                                             
    print ch, 

Output:

$ python iterators.py
p y t h o n

Iterate over dictionary object

for key, val in {"key1": "value1", "key2": "value2"}.items():                   
    print key, "->", val  

Output:

$ python iterators.py
key2 -> value2
key1 -> value1

String to list conversion

new_list = list("python");                                                      
print new_list;  

Output:

$ python iterators.py
['p', 'y', 't', 'h', 'o', 'n']

Iteration Protocol

Built-in function iter takes an iterable object and returns an iterator.
next() method returns next element in the array, raises StopIteration exception if no next element in list.

new_list = list("python");                                                      
iterator = iter(new_list);                                                      
print iterator.next();                                                          
print iterator.next();                                                          
print iterator.next();                                                          
print iterator.next();                                                          
print iterator.next();                                                          
print iterator.next();                                                          
print iterator.next();   

Output:

$ python iterators.py
p
y
t
h
o
n
Traceback (most recent call last):
  File "iterators.py", line 9, in 
    print iterator.next();
StopIteration

once traverse all the elements in list, throws StopIteration error if try next() method.

Simple Iterators in python

class yrange:                                                                   
    def __init__(self, n):                                                      
        self.i = 0                                                              
        self.n = n                                                              
                                                                                
    def __iter__(self):                                                         
        return self                                                             
                                                                                
    def next(self):                                                             
        if self.i < self.n:                                                     
            i = self.i                                                          
            self.i += 1                                                         
            return i                                                            
        else:                                                                   
            raise StopIteration()   
                                            
                                                                                
r = yrange(4);                                                                  
print r.next();                                                                 
print r.next();                                                                 
print r.next();                                                                 
print r.next();                                                                 
print r.next();                                                                 
print r.next();   

here iterator works like built-in xrange function.
Output:

$ python iterators.py
0
1
2
3
Traceback (most recent call last):
  File "iterators.py", line 22, in 
    print r.next(); 
  File "iterators.py", line 15, in next
    raise StopIteration()
StopIteration

What is generators?

• Generators simplifies creation of iterators.
• Generator is a function that generates a sequence of results instead of a single value.
• When yield statement is executed the function generates a next value in a list.
• Generator is also an iterator but don’t have to worry about the iterator protocol.

Pyhton Generators Example

def yrange(n):                                                                  
    i = 0                                                                       
    while i < n:                                                                
        yield i                                                                 
        i += 1                                                                  
                                                                                
r = yrange(4);                                                                  
print r.next();                                                                 
print r.next();                                                                 
print r.next();                                                                 
print r.next();                                                                 
print r.next();                                                                 
print r.next();  

Output:

$ python iterators.py
0
1
2
3
Traceback (most recent call last):
  File "iterators.py", line 12, in 
    print r.next(); 
StopIteration

Generators: yield and call to next method

yield returns generated value and waits to get next method call. it continues from after yield statement once received next method call.

def yrange(n):                                                                  
    i = 0                                                                       
    while i < n:                                                                
        print "before yield";                                                   
        yield i                                                                 
        print "after yield";                                                    
        i += 1                                                                  
                                                                                
r = yrange(4);                                                                  
print "next: ", r.next();                                                       
print "next: ", r.next();                                                       
print "next: ", r.next();                                                       
print "next: ", r.next();                                                       
print "next: ", r.next();  

Output:

$ python iterators.py
next:  before yield
0
next:  after yield
before yield
1
next:  after yield
before yield
2
next:  after yield
before yield
3
next:  after yield
Traceback (most recent call last):
  File "iterators.py", line 14, in 
    print "next: ", r.next(); 
StopIteration

Python Generators Expression

result = (num*2 for num in range(10))                                           
print result 

Output:

$ python iterators.py
 at 0x7f77a64e77d0>

using next method

incorrect sum value once traversed all elements.

result = (num*2 for num in range(2))                                            
                                                                                
print result.next();                                                            
print result.next();                                                            
print "sum: ", sum(result);  

Output:

$ python iterators.py
0
2
sum:  0

Correct way to use math functions:

result = (num*2 for num in range(2))                                                                                                                         
print "sum: ", sum(result); 

Output:

$ python iterators.py
sum:  2

Python itertools module

Python itertools module in the standard library provides lot of interesting tools to do with iterators.
python example to chain multiple iterators together using itertools chain method.

import itertools;                                                               
iterator1 = iter([1, 2, 3, 4])                                                  
iterator2 = iter(['A', 'B', 'C'])                                               
print itertools.chain(iterator1, iterator2);                                    
                                                                                
print list(itertools.chain(iterator1, iterator2)); 

Output:

$ python iterators.py

[1, 2, 3, 4, 'A', 'B', 'C']

enumerate method generator

print enumerate(["one", "two", "three"]);                                               
print list(enumerate(["one", "two", "three"]));  

Output:

$ python iterators.py

[(0, 'one'), (1, 'two'), (2, 'three')]

« Previous Next »