Web Scraping And Api Fundamentals In Python
Last updated 6/2020
MP4 | Video: h264, 1280×720 | Audio: AAC, 44.1 KHz
Learn Web Scraping with Beautiful Soup and requests-html; harness APIs whenever available; automate data collection!
What you’ll learn
Learn the fundamentals of Web Scraping
Implement APIs into your applications
Master working with Beautiful Soup
Create functioning scrapers
Scrape JavaScript
Familiarize yourself with HTML
Get the hang of CSS Selectors
Make HTTP requests
Understand website cookies
Explore scraping content locked behind a log-in system
Limit the rate of requests
Requirements
Python 3 and the Anaconda distribution
Basic Python knowledge
Curiosity and enthusiasm to learn and practice
Description
Overview
Section 1: Introduction to the course
Lecture 1 What does the course cover?
Lecture 2 What is Web Scraping?
Lecture 3 Ethics of Scraping
Lecture 4 Download All Resources
Section 2: Setting up the environment
Lecture 5 Setting up the environment – Do not skip, please!
Lecture 6 Why Python and why Jupyter?
Lecture 7 Installing Anaconda
Lecture 10 Installing the packages
Section 3: Working with APIs
Lecture 11 API overview
Lecture 12 HTTP requests: GET and POST requests
Lecture 13 JSON: preferred data exchange format for APIs
Lecture 14 Exchange rates API: GETting a JSON reply
Lecture 15 Incorporating parameters in a GET request
Lecture 16 Additional API functionalities
Lecture 17 Creating a simple currency converter
Lecture 18 iTunes API
Lecture 19 iTunes API: Exercise
Lecture 20 iTunes API: Structuring and exporting the data
Lecture 21 APIs: Exercise
Lecture 22 GitHub API: Pagination
Lecture 23 EDAMAM API: Initial setup and registration
Lecture 24 EDAMAM API: Sending a POST request
Lecture 25 Downloading files with requests
Section 4: HTML overview
Lecture 26 What is HTML?
Lecture 27 Structure of HTML
Lecture 28 Syntax of HTML. Tags
Lecture 29 Tag attributes
Lecture 30 Popular tags
Lecture 31 CSS and JavaScript
Lecture 32 Character encoding
Lecture 33 XHTML and code style
Section 5: Web Scraping with Beautiful Soup
Lecture 34 Introduction to the Beautiful Soup package
Lecture 35 Workflow of Web Scraping
Lecture 36 Setting up your first scraper
Lecture 37 Searching and navigating the HTML tree
Lecture 38 Searching the HTML tree by attributes
Lecture 39 Extracting data from the HTML tree
Lecture 40 Extracting text from an HTML tag
Lecture 41 Practical example: dealing with links
Lecture 42 Practical example: Exercise
Lecture 43 Extracting data from nested HTML tags
Lecture 44 Scraping multiple pages automatically
Section 6: Practical project: Scraping Rotten Tomatoes
Lecture 45 Setting up your scraper
Lecture 46 Extracting the title and year of each movie
Lecture 47 Extracting the score of each movie: Exercise
Lecture 48 Extracting the rest of the information
Lecture 49 Dealing with the cast of the movies
Lecture 50 Extracting the rest of the information: Exercise
Lecture 51 Storing and exporting the data in a structured form
Section 7: Scraping HTML tables
Lecture 52 Scraping HTML tables with the help of Pandas
Section 8: Practical projects
Lecture 53 Scraping Steam
Lecture 54 Scraping YouTube
Section 9: Common roadblocks when scraping
Lecture 55 Common roadblocks when Web Scraping.
Section 10: The requests-html package
Lecture 56 Introduction to the requests-html package
Lecture 57 Exploring the capabilities of requests-html for Web Scraping
Lecture 58 Searching for text
Lecture 59 CSS selectors
Lecture 60 Scraping JavaScript
Lecture 61 Scraping JavaScript: Exercise
Lecture 62 Completing 100%
You should take this course if you want to learn how to use APIs,This course is for you if you want to learn how to scrape websites,Anyone who wants to learn how to automate the boring and mundane everyday tasks,Individuals who are curious and passionate about data,The course is ideal for beginners to programming who want to learn Beautiful Soup and requests-html
