Last modified: Aug 23, 2023 By Alexander Williams

How to Find Next Element or Elements in Beautifulsoup

Are you having trouble finding the next element or elements with Beautifulsoup? If so, In this article, you will learn an easy method to find the next element using find_next() and find_all_next().

Find Next Element using find_next()

The find_next() method is a feature provided by BeautifulSoup used to navigate through the HTML document's structure and find the next occurrence of a specified HTML element or tag.

Here's the basic syntax of the find_next() method:

element.find_next(name, attrs, text, **kwargs)
  • name: The name of the HTML tag.
  • attrs: A dictionary containing attributes and their values to match against.
  • text: The text content you want the found element to contain.
  • **kwargs: Additional keyword arguments for advanced searches.

In the following example, we'll see how to use find_next():

from bs4 import BeautifulSoup

# HTML content
html = """
<html>
<head>
<title id="title">This is the title</title>
</head>
<body>
    <h1>This is the heading</h1>
    <p>This is the paragraph</p>
    <p>This is the paragraph 2</p>
    <h2>This is the heading</h2>
</body>
</html>
"""

# BeautifulSoup object
soup = BeautifulSoup(html, 'html.parser')

# Find the first <h1> tag
h1 = soup.find("h1")

# Find the next element
next_element = h1.find_next()

# Print the next element
print(next_element)

Output:

<p>This is the paragraph</p>

As you can see, the code above finds the first element of the <h1> tag. We'll see that in the next part if you want to get all the next elements.

You can also set the specific next tag that you want to find.

from bs4 import BeautifulSoup

# HTML content
html = """
<html>
<head>
<title id="title">This is the title</title>
</head>
<body>
    <h1>This is the heading</h1>
    <p>This is the paragraph</p>
    <p>This is the paragraph 2</p>
    <h2>This is the heading</h2>
</body>
</html>
"""

# BeautifulSoup object
soup = BeautifulSoup(html, 'html.parser')

# Find the first <h1> tag
h1 = soup.find("h1")

# Find the next <h2> element
next_element = h1.find_next("h2")

# Print the next element
print(next_element)

Output:

<h2>This is the heading</h2>

Just a quick reminder, if the next element doesn't exist, the find_next() function will return None.

Find All Next Element using find_all_next()

The find_all_next() method is a feature provided by BeautifulSoup used to navigate through the HTML document's structure and find all the next occurrences of a specified HTML element or tag. The function returns the search results as a list.

Here's the syntax of find_all_next()

find_all_next(name, attrs, string, limit, **kwargs)
  • name: The name of the HTML tag you're searching for.
  • attrs: A dictionary containing attributes and their values to match against.
  • string: The string content you want the found elements to contain.
  • limit: An integer that limits the number of elements to be found.
  • **kwargs: Additional keywords for more advanced searches.

In the following example, we will find all the next elements of the <h1> tag.

from bs4 import BeautifulSoup

# HTML content
html = """
<html>
<head>
<title id="title">This is the title</title>
</head>
<body>
    <h1>This is the heading</h1>
    <p>This is the paragraph</p>
    <p>This is the paragraph 2</p>
    <h2>This is the heading</h2>
</body>
</html>
"""

# BeautifulSoup object
soup = BeautifulSoup(html, 'html.parser')

# Find the first <h1> tag
h1 = soup.find("h1")

# Find the all next elements
next_elements = h1.find_all_next()

# Print the next elements
print(next_elements)

Output:

[<p>This is the paragraph</p>, <p>This is the paragraph 2</p>, <h2>This is the heading</h2>]

If you would like to print every element, refer to the code below:

for element in next_elements:
    print(element)

Output:

<p>This is the paragraph</p>
<p>This is the paragraph 2</p>
<h2>This is the heading</h2>

Please note that the find_all_next() method will return an empty list if the next element does not exist.

Conclusion

As you can see in this article, you can easily find the next element or elements using Beautiful Soup by employing the find_next() and find_all_next() methods. 

The first method is used to find the first element, and the second is used to find all the next elements.

Related Tutorials:

Recent Tutorials: