Video covers - What are different triggers available for Spark Streaming? How Processing Time and Once trigger are different? How we can tune Kafka jobs with Partitions?
Chapters:
00:00 - Introduction
00:58 - Automating Device data for Kafka
02:37 - Trigger mode Once/AvailableNow
04:36 - Trigger mode ProcessingTime
05:49 - Tune Kafka Streaming Job
08:23 - Trigger mode Continuous
URLs:
Github Code - github.com/subhamkharwal/spar...
Device data samples - github.com/subhamkharwal/spar...
To setup Kafka with Spark in Local environment - • 03 Spark Streaming Loc...
JSON data Flattening and reading from files - • 07 Spark Streaming Rea...
Keywords: Apache Spark, PySpark, Spark Streaming, Real-time Data Processing, Data Streaming, Big Data Analytics, PySpark Tutorial, Apache Spark Tutorial, Streaming Analytics, Spark Structured Streaming, PySpark Streaming, Big Data Processing.
New video in every 3 days ❤️
Make sure to like and Subscribe.
Негізгі бет Ғылым және технология 11 Spark Streaming Triggers - Once, Processing Time & Continuous | Tune Kafka Streaming Performance
Пікірлер: 6