Seriously it's very comprehensive ,crisp and clear
@ferrerolounge1910
Жыл бұрын
Never seen anyone explain things this easily! wonderful keep it coming! 👍
@sid5201
2 жыл бұрын
Have gone thru similar videos explaining the apache spark architecture, but this has to be the best one. Very comprehensive and clear.
@roshnigaddam7526
2 жыл бұрын
Stumbled upon this channel while preparing for an interview. I am sure I am going to be very confident after watching this play series. Amazing content. Detailed explanation. Thank you!
@mannykhan7752
5 ай бұрын
Your videos are so well-detailed and explained with great clarity. Databricks is a tricky skill to master but your videos make it very easy. Great job.
@elevated_minds09
Жыл бұрын
Hi Bhawna, I just wanted to say thank you for creating such an amazing playlist. Your explanations are so clear and easy to understand, and I really appreciate the effort you put into breaking down these complex topics. I'm working on my new project involving Databricks for machine learning, and your videos have been a lifesaver. I have a question: How is the number of tasks assigned to each core determined at each stage? Is there a default value for this?
@pokeshoot
Жыл бұрын
Really nice content Bhuvana. Apprciates all your hard work behind.
@karthickj8045
2 жыл бұрын
Amazing series of vidoes. Thank you
@raghavendrareddy4765
2 жыл бұрын
Awesome Series Nice explanation
@sriharig9096
2 жыл бұрын
Excellent explanation.....Thank you
@tatianecorrea4277
2 жыл бұрын
Great explanation....You have a lot of didactics!
@yolagatiudaykumarreddy6164
2 жыл бұрын
hey bhawna, videos are really help me a lot.. keep continue and create one more playlist for Realtime concepts in ADB
@sravankumar1767
2 жыл бұрын
In our current project we are using Delta lakes, we are Raw, Trusted, refined, provisioned, provisioned to extract. Raw to trusted- Data quality check good data goes to refined then we apply transformations in refined and provisioned layer. Provisioned to extract we have simple select statements. But we have Day 0- full load and Day 1 - Incremental loading. But I didn't get a chance to work on Day 1. Here we have created the Metadata scripts tables according to that we give job names, elt-cfg, lkup db, Metadata lkup. We have created 3 scripts for each job. In our case we are not writing any merge statement for Incremental loading. How can we find difference b/w full load and incremental loading in Delta lakes. Here Metadata scripts also same for full load and incremental loading. Is there any extra columns are available for Day 1- Incremental loading. Can u pls clarify my doubts
@krishnakumar-vi3tf
Жыл бұрын
You are excellent,I have one doubt ..I am partitioning on date so it creates huge partitions as days goes ,how does it process does it process each partition by a core? what is the mechanism it follows when I have hundreds of partitions? Greatly appreciate your reply !!
@lifechamp007
2 жыл бұрын
Super helpful - Thank you so much !! #StayBlessednHappy
@ankushpatil1114
3 ай бұрын
Thank you so much
@user-vq5ju2uk7j
2 жыл бұрын
Amazing video. good detailed explanation. could you please do a video in-depth explanation about RDD. and i have one doubt in RDD. if we create many rdds and it is stored in memory then will it occupy most of the memory then will we get memory out of exception . how many rdds stored in memory? please expalain Thanks
@sravankumar1767
2 жыл бұрын
Nice explanation 👌 👍
@lovishaghi4674
2 жыл бұрын
Amazing video....Kindly create videos on unit testing as well in databricks using python.
@jdisunil
2 жыл бұрын
Thank you so much from the data engineering community for the great videos you are putting in. One question: are we saying until a display(df) is done the previous commands are not actually actioned? am a new bee. please correct me
@payalgupta1202
Жыл бұрын
yes
@sivagssri
Жыл бұрын
Good video, can you please do video about how to handle scd2 using data bricks. I got this ques in one of interview.
@caiyu538
Жыл бұрын
Great if you show a code how to split the task in cluster manager, it will be easy for us to understand. Pyspark is more powerful than pandas, one of reason is that it can do tasks parallels.
@neelbanerjee7875
Жыл бұрын
One query - No of task = no of cores in an executor? Or No of task = no of partition defined? Can please explain this relations among task, partition and core with an example?
@krishnakumar-vi3tf
Жыл бұрын
Yes I have the same doubt ,does each partition reaches to each core ? in that case I am portioning the data based on date so as days goes number of partitions increase so how this is distributed to each core if I have hundreds or thousands partitions 🤔🤔
@nagamanickam6604
Жыл бұрын
Thank you
@harishkonakandla
2 жыл бұрын
examples when task comes into picture?
@jaydeeppatidar4189
Жыл бұрын
This video is not for beginner. If you already have knowledge of jobs,stages and tasks then it will be helpful only.
@shreyaspurankar9736
Жыл бұрын
Can you provide ppt to me?
@jaydeeppatidar4189
Жыл бұрын
If you explain it like this in words only then no one will understand. Take the EC2 instances and expalin in detail how the storage has been divided. This video can be truncated to 3 parts each in 20min size but you have completed everything in 24 min. only. This concept is very important and big so can't explain it in just 24 min.
Пікірлер: 30