Web Scraping using Selenium and BeautifulSoup libraries in python

Scraping Data from various websites

Photo by Joshua Sortino on Unsplash
  • Beautiful Soup.
  • Scrapy.
  • Selenium.
  • Python Requests.
  • LXML.
  • MechanicalSoup.
  • Urllib

Beautiful Soup

conda install beautifulsoup4pip install BeautifulSoup4
from bs4 import BeautifulSoup as bs4
import requests
url='https://en.wikipedia.org/wiki/Web_scraping'
data=requests.get(url)
soup=bs4(data.text,'html.parser')
print(soup)

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store