Python Requests: Easy Guide to Download Files Like a Pro

Downloading files is a common task in web development, and Python's Requests library makes it straightforward and efficient. Before diving in, ensure you have Requests properly installed on your system.

Basic File Download

The simplest way to download a file using Requests involves making a GET request and writing the content to a local file. Here's a basic example:

import requests

url = "https://example.com/file.pdf"
response = requests.get(url)

# Save the file
with open("downloaded_file.pdf", "wb") as file:
    file.write(response.content)

Streaming Large Files

When dealing with large files, it's better to use streaming to avoid memory issues. The stream parameter in requests.get() helps manage memory efficiently:

import requests

url = "https://example.com/large_file.zip"
response = requests.get(url, stream=True)

# Download with chunks
with open("large_file.zip", "wb") as file:
    for chunk in response.iter_content(chunk_size=8192):
        if chunk:
            file.write(chunk)

Progress Tracking

When downloading large files, it's useful to track progress. Here's how to implement a progress bar using the Content-Length header:

import requests
from tqdm import tqdm

url = "https://example.com/large_file.zip"
response = requests.get(url, stream=True)
total_size = int(response.headers.get('content-length', 0))

with open("large_file.zip", "wb") as file, tqdm(
    desc="Downloading",
    total=total_size,
    unit='iB',
    unit_scale=True
) as progress_bar:
    for data in response.iter_content(chunk_size=1024):
        size = file.write(data)
        progress_bar.update(size)

Error Handling

It's important to implement proper error handling when downloading files. Here's a robust example that handles common issues:

import requests
from requests.exceptions import RequestException

def download_file(url, filename):
    try:
        response = requests.get(url, stream=True)
        response.raise_for_status()  # Raises an HTTPError for bad responses
        
        with open(filename, "wb") as file:
            for chunk in response.iter_content(chunk_size=8192):
                if chunk:
                    file.write(chunk)
        return True
    except RequestException as e:
        print(f"Error downloading file: {e}")
        return False

Verifying Downloads

To ensure file integrity, you can verify downloads using checksums. Here's an example using SHA-256:

import hashlib
import requests

def verify_download(url, filename, expected_sha256):
    # Download file
    response = requests.get(url)
    with open(filename, "wb") as file:
        file.write(response.content)
    
    # Calculate SHA-256
    sha256_hash = hashlib.sha256()
    with open(filename, "rb") as file:
        for chunk in iter(lambda: file.read(4096), b""):
            sha256_hash.update(chunk)
    
    return sha256_hash.hexdigest() == expected_sha256

Best Practices

When downloading files with Python Requests, follow these best practices:

1. Always use stream=True for large files

2. Implement proper error handling

3. Use progress tracking for better user experience

4. Verify file integrity when necessary

Conclusion

The Python Requests library provides powerful tools for downloading files. By following this guide and implementing proper error handling, you can create robust file download functionality in your applications.

For more advanced usage, check out how to handle SSL verification or make asynchronous requests.