Hey, I don'tt get the difference beetwenn stream processing and batchprocessing, here for me this code is streamprocessing and not batch ?due to the readstream and writestream
@thedataguygeorge
3 ай бұрын
The difference in my mind is if you are processing more than one data point at a time, so this example is batch because it waits to collect X number of data points before processing them as a batch
@ytwizard69
3 ай бұрын
Hi I wanna know. If I am writing the data from mysql and writing back to MySQL to another DB. I have more than 20 mill records. pyspark job just failing to even read that data. how do I resolve that.
@thedataguygeorge
3 ай бұрын
Does your Spark cluster have enough memory to hold those rows?
@sjsh3hee815
6 ай бұрын
can you make Kafka tutorial from beginner level?
@thedataguygeorge
6 ай бұрын
Definitely, what would you want to see covered in that?
Пікірлер: 10