What types of exceptions should you catch?

6 min read Original article ↗

What types of exceptions should you catch?

Catching many exceptions at once

When catching an exception, it's generally considered a good idea to only catch exceptions if you understand their origin.

Here we have some code that catches many exception types at once. We're catching a ValueError, a TypeError, a KeyError, and a NameError exception:

import csv
import datetime
import sys

def parse_date(date_string):
    return datetime.date.fromisoformat(date_string)

[filename] = sys.argv[1:]
with open(filename) as csv_file:
    reader = csv.DictReader(csv_file)
    for n, row in enumerate(reader, start=1):
        name = row["name"]
        try:
            start, end = parse_date(row["start"]), parse_date(row["end"])
        except (ValueError, TypeError, KeyError, NameError) as e:
            error = type(e).__name__
            print(f"{error}: Invalid date on line {n}", file=sys.stderr)
            continue
        time = end - start
        print(f"{name}: {time.days} days")

It's not entirely clear why it catches each of these types of exceptions.

When will a NameError be raised?

We probably shouldn't be catching a NameError exception at all because NameError exceptions are usually raised when a variable name has been misspelled. Catching a NameError wouldn't catch a bug in data from an end user; it would suppress a bug in our code!

So we will stop catching a NameError exception immediately.

        except (ValueError, TypeError, KeyError) as e:

When will a ValueError be raised?

A ValueError exception can be raised if our CSV file has some invalid data in it, specifically if any of our rows contains an invalid date.

For example this CSV file has an invalid date (2026-00-01) on line 4:

name,start,end
2021-Q1,2025-01-01,2025-04-01
2021-Q2,2025-04-01,2025-07-01
2021-Q3,2025-07-01,2025-10-01
2021-Q4,2025-10-01,2026-00-01

And when we run our program against it, a ValueError is raised:

$ python3 find_data_periods.py data1.csv
2021-Q1: 90 days
2021-Q2: 91 days
2021-Q3: 92 days
ValueError: Invalid date on line 4

When will a TypeError be raised?

If we have missing dates in our CSV file a TypeError will be raised.

We have a couple of rows in our CSV file that simply don't have some columns:

name,start,end
2021-Q1,2025-01-01,2025-04-01
2021-Q2
2021-Q3,2025-07-01,2025-10-01
2021-Q4,2025-10-01

The start and end columns are missing on some rows in this file, so a TypeError is raised:

$ python3 find_data_periods.py data2.csv
2021-Q1: 90 days
TypeError: Invalid date on line 2
2021-Q3: 92 days
TypeError: Invalid date on line 4

When will a KeyError be raised?

A KeyError is raised in kind of an odd situation: if our CSV file has a typo in a header a KeyError will be raised.

The column start is misspelled in this CSV file (it has an uppercase S instead of lowercase s):

name,Start,end
2021-Q1,2025-01-01,2025-04-01
2021-Q2,2025-04-01,2025-07-01
2021-Q3,2025-07-01,2025-10-01
2021-Q4,2025-10-01,2026-01-01

When we run our program against this file, it seems like there's a problem in every row in our file:

$ python3 find_data_periods.py data3.csv
KeyError: Invalid date on line 1
KeyError: Invalid date on line 2
KeyError: Invalid date on line 3
KeyError: Invalid date on line 4

But there's not really. The problem is just really with our header.

So it might actually be better if we didn't catch a KeyError exception at all. Because the traceback that would be shown to the end user would actually be more clear in this case.

$ python3 find_data_periods.py data3.csv
Traceback (most recent call last):
  File "/home/trey/find_data_periods.py", line 14, in <module>
    start, end = parse_date(row["start"]), parse_date(row["end"])
KeyError: 'start'

Tracebacks are never pretty, but at least this one shows that there's a problem involving the word start. A user could hopefully figure out from this that the lowercase s in the traceback and the uppercase S in our CSV file are mismatched.

This traceback isn't particularly clear, but it is more clear than the error message we were showing before. Sometimes exceptions are better left unhandled.

Handling problems preemptively

Instead of not catching a KeyError at all, we could handle this problem in our headers before we even start looping. Remember, the issue isn't with each row in our file: the problem is just with the first row.

So we'll check our headers line for missing headers. If we have a missing header, we'll print out an error message and then exit our program.

import csv
import datetime
import sys

def parse_date(date_string):
    return datetime.date.fromisoformat(date_string)

[filename] = sys.argv[1:]
with open(filename) as csv_file:
    reader = csv.DictReader(csv_file)
    for header in ["name", "start", "end"]:
        if header not in reader.fieldnames:
            print(f"Error: Missing {header} header", file=sys.stderr)
            sys.exit(1)
    for n, row in enumerate(reader, start=1):
        ...
        time = end - start
        print(f"{name}: {time.days} days")

This is a lot friendlier for our end users:

$ python3 find_data_periods.py data3.csv
Error: Missing start header

You really shouldn't catch all possible exceptions that your program could raise. Some exceptions are better left unhandled, or better handled preemptively, without any exception handling at all.

When should you catch all possible types of exceptions?

There are sometimes cases for catching exceptions without specifying very specific exception types. Sometimes it makes sense to cast a wide net and catch all possible types of exceptions that could be raised.

For example, in this code we're looping over lots of file paths:

from argparse import ArgumentParser
from pathlib import Path
import sys

parser = ArgumentParser()
parser.add_argument("paths", type=Path, nargs="+")
args = parser.parse_args()
for path in args.paths:
    try:
        with open(path) as f:
            line_count = sum(1 for line in f)
        print(f"{path}: {line_count} lines")
    except Exception as e:
        print(f"ERROR READING {path}: {e}", file=sys.stderr)

If an error occurs while we process one of these files counting the lines within it, we don't want to stop our loop suddenly and end our program. Instead, we'd like to print out the error that occurred while processing one of the files and then continue onward to the next file.

This is a much friendlier user experience than the alternative of stopping our whole program:

$ python3 count_lines.py log1.txt log2.txt log3.txt log4.txt log5.txt
log1.txt: 1253 lines
ERROR READING log2.txt: [Errno 13] Permission denied: 'log2.txt'
log3.txt: 550 lines
log4.txt: 1224 lines
ERROR READING log5.txt: 'utf-8' codec can't decode byte 0x81 in position 155: invali
d start byte

We're doing this by catching any possible exception type that could be raised (with the exception of system-exiting exceptions).

This is one of those situations where it might actually make sense to do this, because this is somewhat mission-critical code; that is code that we wouldn't want to halt our program simply because an error occurred.

Summary

The trickiest programming bugs are often caused by catching exceptions that you didn't mean to catch, or handling exceptions in ways that obfuscate the actual error that's occurring.

In general, you should try to make sure that you're only catching specific exceptions. The specific exceptions that your code might raise, and that you understand when your code would raise such an exception.

The one big exception to this is mission-critical parts of your code. For mission-critical code, you might want to catch all possible types of exceptions that might be raised and log them for later evaluation.