Elasticsearch Python API Guide for Beginners

Elasticsearch is a powerful search engine. It is built on Apache Lucene. It allows for fast, full-text search and analytics. Many developers use it for log analysis, product search, and data exploration.

To interact with Elasticsearch from Python, you use the official Elasticsearch Python client. This API provides a simple way to index, search, and manage your data. This guide will show you how to get started.

We will cover installation, basic operations, and common tasks. By the end, you will be able to use Elasticsearch in your Python projects.

Installation and Setup

First, you need to install the Elasticsearch Python client. Use pip, the Python package manager. The command is simple.

pip install elasticsearch

Make sure you have a running Elasticsearch instance. You can run it locally or connect to a cloud service. For local development, Docker is a good option.

Once installed, import the library and create a client. The client connects to your Elasticsearch server. Here is the basic setup.

from elasticsearch import Elasticsearch

# Connect to localhost:9200 by default
es = Elasticsearch()

# For a specific host, use:
# es = Elasticsearch(["http://localhost:9200"])

This creates a client connected to `localhost:9200`. You can now use the es object to interact with Elasticsearch.

Indexing Your First Document

Elasticsearch stores data in indices. An index is like a database in SQL. You add documents to an index. A document is a JSON object.

To index a document, use the index method. You must specify an index name and a document body. Each document gets a unique ID.

# Index a simple document
document = {
    "title": "Getting Started with Elasticsearch",
    "content": "This is a guide for beginners.",
    "date": "2023-10-26"
}

response = es.index(index="blog_posts", id=1, body=document)
print(response)

{'_index': 'blog_posts', '_id': '1', '_version': 1, 'result': 'created', '_shards': {'total': 2, 'successful': 1, 'failed': 0}, '_seq_no': 0, '_primary_term': 1}

The response confirms the document was created. The `result` field shows `'created'`. The document is now stored in the `blog_posts` index with ID `1`.

Indexing is a core operation. For more on handling data in Python, see our Python API Data Pulling Guide.

Searching for Documents

Searching is where Elasticsearch shines. Use the search method. You can perform simple or complex queries. Let's search for our document.

# Search for documents in the index
query = {
    "query": {
        "match_all": {}  # Match all documents
    }
}

response = es.search(index="blog_posts", body=query)
print(f"Found {response['hits']['total']['value']} hits.")

for hit in response['hits']['hits']:
    print(f"ID: {hit['_id']}, Title: {hit['_source']['title']}")

Found 1 hits.
ID: 1, Title: Getting Started with Elasticsearch

The search returns a list of hits. Each hit contains the document data. You can access it via `hit['_source']`.

For more precise queries, use a match query. This searches within specific fields.

# Search for documents containing "beginners" in the content
query = {
    "query": {
        "match": {
            "content": "beginners"
        }
    }
}

response = es.search(index="blog_posts", body=query)
print(f"Hits for 'beginners': {response['hits']['total']['value']}")

Hits for 'beginners': 1

This matches documents where the `content` field contains the word "beginners". Elasticsearch uses powerful text analysis for this.

Updating and Deleting Documents

You can update existing documents. Use the update method. Provide the index, ID, and the changes. The changes are in a `doc` field.

# Update the document's title
update_body = {
    "doc": {
        "title": "Updated Elasticsearch Guide"
    }
}

response = es.update(index="blog_posts", id=1, body=update_body)
print(response['result'])

updated

To delete a document, use the delete method. Specify the index and ID.

# Delete the document
response = es.delete(index="blog_posts", id=1)
print(response['result'])

deleted

These operations help manage your data lifecycle. They are essential for maintaining your indices.

Handling Index Settings and Mappings

Indices have settings and mappings. Settings control behavior like shard count. Mappings define the data types for fields. You can create an index with custom mappings.

# Define a mapping for a new index
mapping = {
    "mappings": {
        "properties": {
            "title": {"type": "text"},
            "content": {"type": "text"},
            "date": {"type": "date"},
            "view_count": {"type": "integer"}  # Numeric field
        }
    }
}

# Create the index with the mapping
es.indices.create(index="articles", body=mapping)

This creates an `articles` index. The `view_count` field is an integer. For details on numeric data, check our Python API Number Handling Guide.

You can also check if an index exists.

# Check index existence
exists = es.indices.exists(index="articles")
print(f"Index exists: {exists}")

Index exists: True

Error Handling and Best Practices

The Elasticsearch client can raise exceptions. Always handle them. Common errors include connection issues and missing indices.

from elasticsearch import Elasticsearch, NotFoundError

es = Elasticsearch()

try:
    es.get(index="missing_index", id=999)
except NotFoundError:
    print("Document or index not found.")
except ConnectionError:
    print("Could not connect to Elasticsearch.")

Use connection pooling for production. Set up multiple hosts for reliability. Here is a more robust client setup.

es = Elasticsearch(
    ["http://node1:9200", "http://node2:9200"],
    sniff_on_start=True,
    sniff_on_connection_fail=True,
    sniffer_timeout=60
)

This configuration helps with high availability. The client can sniff the cluster for nodes.

For automating