Best HandsOn Big Data Practices With Pyspark & Spark Tuning

Best Hands-On Big Data Practices With Pyspark & Spark Tuning
Last updated 8/2022
MP4 | Video: h264, 1280×720 | Audio: AAC, 44.1 KHz

[NEW] Semi-Structured (JSON), Structured and Unstructured Data Analysis with Spark and Python & Spark Performance Tuning

What you’ll learn
Understand Apache Spark’s framework, execution and programming model for the development of Big Data Systems
Learn how to work with a free Cloud-based and a Desktop machine for Spark setup and configuration
Learn step-by-step hands-on PySpark practices on structured, unstructured and semi-structured data using RDD, DataFrame and SQL
Investigate and apply optimization and performance tuning methods to manage data Skewness and prevent Spill
Investigate and apply Adaptive Query Execution (AQE) to optimize Spark SQL query execution at runtime
Requirements
Very basic Python and SQL
If you are new to Python programming, Don’t worry at all, you can learn it freely through my YouTube channel. Subscribe to my YouTube channel and keep learning without any hassle
Description
Overview

Section 1: Introduction to Course

Lecture 1 Learn Hands-on Python on my YouTube Channel (free)

Lecture 2 We Would Like to Know What You Think!

Lecture 3 PySpark for Parallel Processing

Lecture 4 Spark Coding Environment

Section 2: PySpark for a large Semi-Structured (JSON) File

Lecture 9 JSON analysis using RDD

Lecture 10 JSON analysis using DF

Section 3: PySpark for a large Structured File

Lecture 11 Structured Data Analysis using RDD

Lecture 12 Structured Data Analysis using DF

Section 4: PySpark for a large Unstructured (LOG) File

Lecture 13 RDD for Log File Analysis

Section 5: Distributed Processing Challenges and Spark Performance Tuning (Practices)

Lecture 17 Spark Optimization for Better Performance (Prevent Spill)

Lecture 18 Spark Optimization using Adaptive Query Execution_1

Lecture 19 Spark Optimization using Adaptive Query Execution_2

Beginner/Junior/Senior Data Developers who want to master Spark/PySpark and Spread the knowledge of Big Data Analytics,If you are new to Python programming, Don’t worry at all, you can learn it freely through my YouTube channel. Subscribe to my YouTube channel and keep learning without any hassle