Web Scraping using Selenium and BeautifulSoup libraries in python

Scraping Data from various websites

  • Scrapy.
  • Selenium.
  • Python Requests.
  • LXML.
  • MechanicalSoup.
  • Urllib

Beautiful Soup

Beautiful Soup is a Python package for parsing HTML and XML documents.It is available for Python 2.7 and Python 3.

conda install beautifulsoup4pip install BeautifulSoup4
from bs4 import BeautifulSoup as bs4
import requests
url='https://en.wikipedia.org/wiki/Web_scraping'
data=requests.get(url)
soup=bs4(data.text,'html.parser')
print(soup)