8. Exception handling

8.1. Introduction

Exception handling is the process of handling the run time error. Error may occur due to missing data and invalid data etc. These error can be catch in the runtime using try-except block and then can be processed according to our need. This chapter presents some of the examples of error handling.

For this first create a new file with missing data in it, as shown below. Here ‘price’ column is empty for Silver,

1
2
3
4
5
6
7
8
9
$ cat price_missing.csv

date,metal,radius,price,quantity
"2016-06-12","Gold",5.5,80.99,1
"2015-07-13","Silver",40.3,,3
"2016-01-21","Iron",9.2,14.29,8
"2014-03-23","Gold",8,120.3,2
"2017-09-11","Copper",4.1,70.25,12
"2011-01-20","Iron",3.25,10.99,3

Now try to calculate the total price for this file using ‘ring_cost’ function. A ValueError will be displayed as shown below,

>>> from price import ring_cost
>>> ring_cost('price_missing.csv')
Traceback (most recent call last):
    [...]
    row[3] = float(row[3]) # price
ValueError: could not convert string to float:

8.2. try-except block

The problem discussed in above section can be solved using try-except block. In this block, the ‘try’ statement can be used to try the string to float/int conversion; and if it fails then ‘except’ block can be used to skip the processing of that particular row, as shown below,

 # price.py

 import csv

 def ring_cost(filename):
     ''' calculate the total cost '''

     total_price = 0 # for all items in the list

     with open(filename, 'r') as f: # open file in read mode
         rows = csv.reader(f)
         header = next(rows) # skip line 1 i.e. header
         for row in rows:
             try:
                 row[3] = float(row[3]) # price
                 row[4] = int(row[4]) # quantity
             except ValueError: # process ValueError only
                 print("Invalid data, row is skipped")
                 continue
             total_price += row[3] * row[4]

     # print("Total price = %10.2f" % total_price)
     return total_price  # return total_price

 def main():
     total = ring_cost('price.csv')  # function call
     print("Total price = %10.2f" % total) # print value

 # standard boilerplate
 # main is the starting function
 if __name__ == '__main__':
     main()

Now process the file again and the processing will skip the invalid line and display the total price.

>>> from price import ring_cost
>>> ring_cost('price_missing.csv')
Invalid data, row is skipped
1311.8799999999999

8.3. Report error

In previous section, the invalid data was ignored and a message was printed. But it is better give some details about the error as well, which is discussed in this section.

8.3.1. Type of error

In the below code, the type of the error is printed on the screen.

# price.py

import csv

def ring_cost(filename):
    ''' calculate the total cost '''

    total_price = 0 # for all items in the list

    with open(filename, 'r') as f: # open file in read mode
        rows = csv.reader(f)
        header = next(rows) # skip line 1 i.e. header
        for row in rows:
            try:
                row[3] = float(row[3]) # price
                row[4] = int(row[4]) # quantity
            except ValueError as err: # process ValueError only
                print("Invalid data, row is skipped")
                print('Reason :', err)
                continue
            total_price += row[3] * row[4]

    # print("Total price = %10.2f" % total_price)
    return total_price  # return total_price

def main():
    total = ring_cost('price.csv')  # function call
    print("Total price = %10.2f" % total) # print value

# standard boilerplate
# main is the starting function
if __name__ == '__main__':
    main()

Note

Do not forget to restart the Python shell after changing the code.

>>> from price import ring_cost
>>> ring_cost('price_missing.csv')
Invalid data, row is skipped
Reason : could not convert string to float:
1311.8799999999999

8.3.2. Location of error

We can use the ‘enumerate’ to display the location of the error.

# price.py

import csv

def ring_cost(filename):
    ''' calculate the total cost '''

    total_price = 0 # for all items in the list

    with open(filename, 'r') as f: # open file in read mode
        rows = csv.reader(f)
        header = next(rows) # skip line 1 i.e. header
        for row_num, row in enumerate(rows, start=1): # start from 1, not 0)
            try:
                row[3] = float(row[3]) # price
                row[4] = int(row[4]) # quantity
            except ValueError as err: # process ValueError only
                print("Invalid data, row is skipped")
                print('Row: {}, Reason : {}'.format(row_num, err))
                continue
            total_price += row[3] * row[4]

    # print("Total price = %10.2f" % total_price)
    return total_price  # return total_price

def main():
    total = ring_cost('price.csv')  # function call
    print("Total price = %10.2f" % total) # print value

# standard boilerplate
# main is the starting function
if __name__ == '__main__':
    main()

Now run the code again and it will display the location of the error,

>>> from price import ring_cost
>>> ring_cost('price_missing.csv')
Invalid data, row is skipped
Row: 2, Reason : could not convert string to float:
1311.8799999999999

8.4. Catch all error (bad practice)

In previous sections, we catch the specific error i.e. ‘ValueError’. We can catch all types of errors as well using ‘Exception’ keyword, but this may result in misleading messages, as shown in this section.

First replace the ‘ValueError’ with ‘Exception’ as below,

# price.py

import csv

def ring_cost(filename):
    ''' calculate the total cost '''

    total_price = 0 # for all items in the list

    with open(filename, 'r') as f: # open file in read mode
        rows = csv.reader(f)
        header = next(rows) # skip line 1 i.e. header
        for row_num, row in enumerate(rows, start=1): # start from 1, not 0)
            try:
                row[3] = float(row[3]) # price
                row[4] = int(row[4]) # quantity
            except Exception as err: # process ValueError only
                print("Invalid data, row is skipped")
                print('Row: {}, Reason : {}'.format(row_num, err))
                continue
            total_price += row[3] * row[4]

    # print("Total price = %10.2f" % total_price)
    return total_price  # return total_price

def main():
    total = ring_cost('price.csv')  # function call
    print("Total price = %10.2f" % total) # print value

# standard boilerplate
# main is the starting function
if __name__ == '__main__':
    main()

Now run the code and it will work fine as shown below,

>>> from price import ring_cost
>>> ring_cost('price_missing.csv')
Invalid data, row is skipped
Row: 2, Reason : could not convert string to float:
1311.8799999999999

Next, replace the ‘int’ with ‘integer’ at Line 16, i.e. we are introducing error in the code.

# price.py

import csv

def ring_cost(filename):
    ''' calculate the total cost '''

    total_price = 0 # for all items in the list

    with open(filename, 'r') as f: # open file in read mode
        rows = csv.reader(f)
        header = next(rows) # skip line 1 i.e. header
        for row_num, row in enumerate(rows, start=1): # start from 1, not 0)
            try:
                row[3] = float(row[3]) # price
                row[4] = integer(row[4]) # quantity
            except Exception as err: # process ValueError only
                print("Invalid data, row is skipped")
                print('Row: {}, Reason : {}'.format(row_num, err))
                continue
            total_price += row[3] * row[4]

    # print("Total price = %10.2f" % total_price)
    return total_price  # return total_price

def main():
    total = ring_cost('price.csv')  # function call
    print("Total price = %10.2f" % total) # print value

# standard boilerplate
# main is the starting function
if __name__ == '__main__':
    main()

Now run the code again and it will give display following messages, which has no relation with the actual error. Actual error is the ‘integer’ at line 16; since the conversion operation can not be performed now (due to invalid type ‘integer’), therefore error is catch for all the rows and the messages are generated for each row. Therefore it is very bad idea to catch errors using “Exception” keyword.

>>> from price import ring_cost
>>> ring_cost('price_missing.csv')
Invalid data, row is skipped
Row: 1, Reason : name 'integer' is not defined
Invalid data, row is skipped
Row: 2, Reason : could not convert string to float:
Invalid data, row is skipped
Row: 3, Reason : name 'integer' is not defined
Invalid data, row is skipped
Row: 4, Reason : name 'integer' is not defined
Invalid data, row is skipped
Row: 5, Reason : name 'integer' is not defined
Invalid data, row is skipped
Row: 6, Reason : name 'integer' is not defined
0

8.5. Silencing error

Sometimes it is undesirable to display the error messages of the except blocks. In such cases, we want to silent the error messages, which is discussed in this section.

First, undo the changes made in previous section, i.e. replace ‘integer’ with ‘int’ and ‘Exception’ with ‘ValueError’.

Now, we will consider the following three cases to handle the error,

  1. silent : do not display error message
  2. warn : display the error message
  3. stop : stop execution of code, if error is detected

For this one positional argument ‘mode’ is defined, whose default value is set to ‘warn’, and then put the ‘print’ statement inside the ‘if-else’ block. It is good idea to set the default value of ‘mode’ to ‘warn’ as we do not want to pass the error silently.

Listing 8.1 Silencing error
# price.py

import csv

# warn is kept as default, as error should not be passed silently
def ring_cost(filename, mode='warn'):
    ''' calculate the total cost '''

    total_price = 0 # for all items in the list

    with open(filename, 'r') as f: # open file in read mode
        rows = csv.reader(f)
        header = next(rows) # skip line 1 i.e. header
        for row_num, row in enumerate(rows, start=1): # start from 1, not 0)
            try:
                row[3] = float(row[3]) # price
                row[4] = int(row[4]) # quantity
            except ValueError as err: # process ValueError only
                if mode == 'warn':
                    print("Invalid data, row is skipped")
                    print('Row: {}, Reason : {}'.format(row_num, err))
                elif mode == 'silent':
                    pass # do nothing
                elif mode == 'stop':
                    raise # raise the exception
                continue
            total_price += row[3] * row[4]

    # print("Total price = %10.2f" % total_price)
    return total_price  # return total_price

def main():
    total = ring_cost('price.csv')  # function call
    print("Total price = %10.2f" % total) # print value

# standard boilerplate
# main is the starting function
if __name__ == '__main__':
    main()

Below are the outputs for each of the cases .. code-block:: python

>>> from price import ring_cost
>>> ring_cost('price_missing.csv')  # default 'warn'
Invalid data, row is skipped
Row: 2, Reason : could not convert string to float:
1311.8799999999999
>>> ring_cost('price_missing.csv', mode='warn')
Invalid data, row is skipped
Row: 2, Reason : could not convert string to float:
1311.8799999999999
>>> ring_cost('price_missing.csv', mode='silent')
1311.8799999999999
>>> ring_cost('price_missing.csv', mode='stop')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/media/dhriti/work/git/advance-python-tutorials/codes/price.py", line 16, in ring_cost
    row[3] = float(row[3]) # price
ValueError: could not convert string to float:

8.6. List of Exception in Python

Following is the list of exceptions available in Python,

BaseException
 +-- SystemExit
 +-- KeyboardInterrupt
 +-- GeneratorExit
 +-- Exception
      +-- StopIteration
      +-- StopAsyncIteration
      +-- ArithmeticError
      |    +-- FloatingPointError
      |    +-- OverflowError
      |    +-- ZeroDivisionError
      +-- AssertionError
      +-- AttributeError
      +-- BufferError
      +-- EOFError
      +-- ImportError
           +-- ModuleNotFoundError
      +-- LookupError
      |    +-- IndexError
      |    +-- KeyError
      +-- MemoryError
      +-- NameError
      |    +-- UnboundLocalError
      +-- OSError
      |    +-- BlockingIOError
      |    +-- ChildProcessError
      |    +-- ConnectionError
      |    |    +-- BrokenPipeError
      |    |    +-- ConnectionAbortedError
      |    |    +-- ConnectionRefusedError
      |    |    +-- ConnectionResetError
      |    +-- FileExistsError
      |    +-- FileNotFoundError
      |    +-- InterruptedError
      |    +-- IsADirectoryError
      |    +-- NotADirectoryError
      |    +-- PermissionError
      |    +-- ProcessLookupError
      |    +-- TimeoutError
      +-- ReferenceError
      +-- RuntimeError
      |    +-- NotImplementedError
      |    +-- RecursionError
      +-- SyntaxError
      |    +-- IndentationError
      |         +-- TabError
      +-- SystemError
      +-- TypeError
      +-- ValueError
      |    +-- UnicodeError
      |         +-- UnicodeDecodeError
      |         +-- UnicodeEncodeError
      |         +-- UnicodeTranslateError
      +-- Warning
           +-- DeprecationWarning
           +-- PendingDeprecationWarning
           +-- RuntimeWarning
           +-- SyntaxWarning
           +-- UserWarning
           +-- FutureWarning
           +-- ImportWarning
           +-- UnicodeWarning
           +-- BytesWarning
           +-- ResourceWarning

8.7. Conclusion

In this chapter, we saw various ways to handle the error along with some good practices. In next chapter, we will discuss the ‘data manipulation’ techniques using various data structures available in Python.