Apache Kafka Series – Learn Apache Kafka For Beginners V3
Last updated 10/2022
MP4 | Video: h264, 1280×720 | Audio: AAC, 44.1 KHz
What you’ll learn
Understand Apache Kafka Ecosystem, Architecture, Core Concepts and Operations
Learn major CLIs: kafka-topics, kafka-console-producer, kafka-console-consumer, kafka-consumer-groups, kafka-configs
Create your Producers and Consumers in Java to interact with Kafka
Program a Real World Twitter Producer & ElasticSearch Consumer
Extended APIs Overview (Kafka Connect, Kafka Streams), Case Studies and Big Data Architecture
Practice and Understand Log Compaction
Requirements
A recent Windows / Mac / Linux machine with minimum 4GB of RAM, 5 GB of disk space
Some understanding of Java Programming
Good to have knowledge about Linux command line
Desire to learn something awesome and new!
Description
Overview
Section 1: Kafka Introduction
Lecture 1 Course Introduction
Lecture 2 Apache Kafka in 5 minutes
Lecture 3 Course Objectives
Lecture 4 Welcome! – About your instructor
Section 2: Code Download
Lecture 5 Code Download
Section 3: ====== Kafka Fundamentals ======
Lecture 6 Kafka Fundamentals
Section 4: Kafka Theory
Lecture 8 Producers and Message Keys
Lecture 9 Consumers & Deserialization
Lecture 10 Consumer Groups & Consumer Offsets
Lecture 11 Brokers and Topics
Lecture 12 Topic Replication
Lecture 13 Producer Acknowledgements & Topic Durability
Lecture 14 Zookeeper
Lecture 15 Kafka KRaft – Removing Zookeeper
Lecture 16 Theory Roundup
Lecture 18 FAQ for Setup Problems
Lecture 20 Mac OS X – Download and Setup Kafka in PATH
Lecture 22 Mac OS X – Using brew
Lecture 23 Linux – Download and Setup Kafka in PATH
Lecture 25 Windows WSL2 – Download Kafka and PATH Setup
Lecture 27 Windows WSL2 – How to Fix Problems
Lecture 28 Windows WSL2 – Extra Instructions
Lecture 30 Note: not ready for production yet
Section 7: CLI (Command Line Interface) 101
Lecture 34 CLI Introduction
Lecture 35 WINDOWS WARNING: PLEASE READ
Lecture 36 Kafka Topics CLI
Lecture 37 Kafka Console Producer CLI
Lecture 38 Kafka Console Consumer CLI
Lecture 39 Kafka Consumers in Group
Lecture 40 Kafka Consumer Groups CLI
Lecture 41 Resetting Offsets
Section 8: Kafka UI
Lecture 42 Conduktor – Demo
Section 9: Kafka Java Programming 101
Lecture 43 Kafka SDK List
Lecture 44 Creating Kafka Project
Lecture 45 Java Producer
Lecture 46 Java Producer Callbacks
Lecture 47 Java Producer with Keys
Lecture 48 Java Consumer
Lecture 49 Java Consumer – Graceful Shutdown
Lecture 50 Java Consumer inside Consumer Group
Lecture 51 Java Consumer Incremental Cooperative Rebalance & Static Group Membership
Lecture 52 Java Consumer Incremental Cooperative Rebalance – Practice
Lecture 53 Java Consumer Auto Offset Commit Behavior
Lecture 54 Programming – Advanced Tutorials
Section 10: ===== Kafka Real World Project =====
Lecture 55 Real World Project Overview
Lecture 56 Real World Exercise – Solution
Section 11: Kafka Wikimedia Producer & Advanced Producer Configurations
Lecture 57 Wikimedia Producer Project Setup
Lecture 58 Wikimedia Producer Implementation
Lecture 59 Wikimedia Producer Run
Lecture 60 Wikimedia Producer – Producer Config Intros
Lecture 61 Producer Acknowledgements Deep Dive
Lecture 62 Producer Retries
Lecture 63 Idempotent Producer
Lecture 64 Safe Kafka Producer Settings
Lecture 65 Wikimedia Producer Safe Producer Implementation
Lecture 66 Kafka Message Compression
Lecture 67 linger.ms and batch.size Producer settings
Lecture 68 Wikimedia Producer High Throughput Implementation
Lecture 70 [Advanced] max.block.ms and buffer.memory
Section 12: OpenSearch Consumer & Advanced Consumer Configurations
Lecture 71 OpenSearch Consumer – Project Overview
Lecture 72 OpenSearch Consumer – Project Setup
Lecture 73 Setting up OpenSearch on Docker
Lecture 74 Setting up OpenSearch on the Cloud
Lecture 75 OpenSearch 101
Lecture 78 Consumer Delivery Semantics
Lecture 80 Consumer Offsets Commit Strategies
Lecture 83 Consumer Offset Reset Behavior
Lecture 85 Consumer Internal Threads
Lecture 86 Consumer Replica Fetching – Rack Awareness
Section 13: Kafka Extended APIs for Developers
Lecture 87 Kafka Extended APIs – Overview
Lecture 88 Kafka Connect Introduction
Lecture 89 Kafka Connect Hands On: Warning
Lecture 90 Kafka Connect Wikimedia & ElasticSearch Hands On
Lecture 91 Kafka Streams Introduction
Lecture 92 Kafka Streams Hands-On
Lecture 93 Kafka Schema Registry Introduction
Lecture 94 Kafka Schema Registry Hands On
Lecture 95 Which Kafka API should I use?
Section 14: Real World Insights and Case Studies (Big Data / Fast Data)
Lecture 97 Kafka Topics Naming Convention
Lecture 98 Case Study – MovieFlix
Lecture 99 Case Study – GetTaxi
Lecture 100 Case Study – MySocialMedia
Lecture 101 Case Study – MyBank
Lecture 102 Case Study – Big Data Ingestion
Lecture 103 Case Study – Logging and Metrics Aggregation
Section 15: Kafka in the Enterprise for Admins
Lecture 104 Kafka Cluster Setup High Level Architecture Overview
Lecture 105 Kafka Monitoring & Operations
Lecture 106 Kafka Security
Lecture 107 Kafka Multi Cluster & MirrorMaker
Lecture 108 Advertised Listeners: Kafka Client & Server Communication Protocol
Section 16: ===== Advanced Kafka =====
Lecture 109 Advanced Kafka
Section 17: Advanced Topics Configurations
Lecture 110 Changing a Topic Configuration
Lecture 111 Segment and Indexes
Lecture 112 Log Cleanup Policies
Lecture 113 Log Cleanup Delete
Lecture 114 Log Compaction Theory
Lecture 115 Log Compaction Practice
Lecture 116 Unclean Leader Election
Lecture 117 Large Messages in Kafka
Section 18: ===== Next Steps =====
Lecture 118 What’s Next?
Lecture 119 THANK YOU!
Lecture 120 Bonus Lecture
