[Python] how to get the title of a web page using BeautifulSoup

Question

I want to fetch the title of web pages using the BeautifulSoup Python package. How can I use this package to get the title of the page?

Answer 1

It's simple to get the title of a web page using the BeautifulSoup Python package. You need to use the "title" attribute of the soup object.

Here is an example:

from bs4 import BeautifulSoup

import urllib.request

#

# user agent so that request is not declined

user_agent = 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.7) Gecko/2009021910 Firefox/3.0.7'

headers = {'User-Agent': user_agent, }

url = "your_url_here"

#

# open the page

request = urllib.request.Request(url, None, headers)

response = urllib.request.urlopen(request)

soup = BeautifulSoup(response, 'html.parser')

#

# fetch the title

print(soup.title.text)

Comments

You can also use the find() function.

foo = soup.find("title")
print(foo.text)