Python Requests: Essential Guide to HTTP Error Handling

When working with requests library in Python, handling HTTP errors and exceptions properly is crucial for building robust applications. This guide will show you how to manage various HTTP errors effectively.

Common HTTP Errors in Requests

While making HTTP requests using requests.get() or other methods like POST, you might encounter various HTTP status codes indicating different types of errors.

Basic Error Handling

Here's a basic example of handling HTTP errors:

import requests

try:
    response = requests.get('https://api.example.com/nonexistent')
    response.raise_for_status()
except requests.exceptions.HTTPError as err:
    print(f"HTTP Error occurred: {err}")
except requests.exceptions.RequestException as err:
    print(f"Error occurred: {err}")

HTTP Error occurred: 404 Client Error: Not Found for url: https://api.example.com/nonexistent

Using Status Code Checks

You can check status codes explicitly to handle different scenarios:

response = requests.get('https://api.example.com/data')
if response.status_code == 200:
    data = response.json()
elif response.status_code == 404:
    print("Resource not found")
elif response.status_code == 500:
    print("Server error occurred")

Handling Timeouts

Timeout errors are common when making HTTP requests. Learn more about handling timeouts in our guide on Python Requests Timeout.

try:
    response = requests.get('https://api.example.com', timeout=5)
except requests.exceptions.Timeout:
    print("The request timed out")

Connection Errors

Handle connection-related errors with specific exception handling:

try:
    response = requests.get('https://nonexistent-site.com')
except requests.exceptions.ConnectionError:
    print("Failed to connect to the server")

JSON Parsing Errors

When working with JSON responses, handle parsing errors properly. For more details, check our guide on handling JSON responses.

try:
    response = requests.get('https://api.example.com/data')
    data = response.json()
except requests.exceptions.JSONDecodeError:
    print("Failed to decode JSON response")

Custom Error Handling

Create a custom error handler for more sophisticated error management:

def handle_request(url, method='get', **kwargs):
    try:
        response = getattr(requests, method)(url, **kwargs)
        response.raise_for_status()
        return response
    except requests.exceptions.HTTPError as http_err:
        print(f"HTTP error occurred: {http_err}")
    except requests.exceptions.ConnectionError as conn_err:
        print(f"Connection error occurred: {conn_err}")
    except requests.exceptions.Timeout as timeout_err:
        print(f"Timeout error occurred: {timeout_err}")
    except requests.exceptions.RequestException as err:
        print(f"An error occurred: {err}")
    return None

# Usage
response = handle_request('https://api.example.com/data')

Best Practices

Always use try-except blocks when making HTTP requests to handle potential errors gracefully.

Implement proper logging instead of just printing errors in production environments.

Set appropriate timeouts to prevent your application from hanging indefinitely.

Conclusion

Proper error handling is essential for building reliable applications that interact with web services. Understanding and implementing these error handling techniques will make your code more robust and maintainable.