New setup? 👀 I am working on next GCP project, I will upload it this month and launch SQL for Data Engineering course too :D Don't forget to like this video :)
@finnyar
Жыл бұрын
hi i am not able to get any internship in data field i have a ba hons degree i have made portfolio also what should i do
@arpitajana7788
Жыл бұрын
Love the new setup! and very very thanks for this amazing info. Excited for SQL course 🤩🤩
@endangeredcreator5050
Жыл бұрын
TL;DR 0:50 AWS S3 (Simple Storage Service ) To store any type of data 1:51 AWS Glue used for transformation scripts 2:29 AWS EMR compute over a distribued cluster 2:55 AWS Redshift data warehouse where extracted datas are loaded for further process 3:30 AWS Athena to directly querry data from S3 without storing it in warehouse 4:39 AWS Lambda compute service 5:26 AWS Kinesis to process data in real time 6:06 AWS DMS (Data Migration Service)
whats the difference between EMR and Lambda as both are for compute purpose?
@haarishkhan61
6 ай бұрын
😊
@hritikapal683
Жыл бұрын
After completing your python course this particular video made lot of sense, still practicing and learning these cloud services thanks man! Btw setup seems nice🔥👏🏻
@DarshilParmar
Жыл бұрын
Awesome Hritika, keep practising and you’ll understand it better
@gavinkalaher7314
Жыл бұрын
Great video Darshil, comprehensive explaination and good scope of DE processes. After I've completed the 2 main courses I'm currently studying, I'll switch focus to some AWS projects.
@DarshilParmar
Жыл бұрын
Thanks Gavin, all the best 👍🏼
@amitjaiswal781
Жыл бұрын
Please also make video on GCP Data Engineer Roadmap.
@amandeepsingh3143
Ай бұрын
Thank you for this informative video
@krightjrar1356
Жыл бұрын
Bro, What do you think what's the future of data engineering after this recent AI boom ? I joined an MNC as a fresher in for data integration role.
@istwishmeister9368
Жыл бұрын
Hi Darshil your videos on Data Engineer are very interesting and to the point. I would like to make a request can you please make videos on Azure. For example showing, completea pipeline for any OLTP or OLAP project? Thanking you in advance
@DarshilParmar
Жыл бұрын
I will put this in my list
@adaparachana7182
9 ай бұрын
I really loved your classes but I have one concern if you make any projects related to streaming data using technologies like Hadoop, Kafka, spark, hive, pig, and Flik related to hospital data it would be very helpful for me as a learner of your videos
@rayyanabdulwajid7681
Жыл бұрын
Book My Show has a crazy ETL pipeline. Thanks for showing us.
@VaibhavShewale
Жыл бұрын
wel that was very useull now i know everything!
@sushantadnaik7548
Жыл бұрын
Great video brother ur all videos are very helpful for me..
@varunmahanot5766
Жыл бұрын
We can even modify data through Athena by using ICEBERG type table.
@DarshilParmar
Жыл бұрын
Yes
@rajkumardubey5486
Ай бұрын
Make video of dms
@jacksmith7160
Жыл бұрын
Darshil in data engineering We should know a lot other than practice Or practice will make us stronger in data engineering
@DarshilParmar
Жыл бұрын
Theory + Practice = Win
@muskanjaiswar2590
Жыл бұрын
Keep it up! It helps a lot for students like us who want to get into data engineering
@DarshilParmar
Жыл бұрын
Thank you Muskan
@yuvraj2913
Жыл бұрын
Hey Pls tell whether it is mandatory to give AWS certification exam to become cloud data engineer ?
@mmp9371
Жыл бұрын
please make video and courses on azure Data engineering road map, thanks.
@hiteshramrakhyani6828
Жыл бұрын
Well explained Great stuff buddy ✌️
@anildangol
Жыл бұрын
Great Video man!
@anuragsoni1509
Жыл бұрын
Hey Darshil, I was waiting for this video to understand the integration of cloud services with ETL process and fundamentals of these services. Thanks for the explanation !!
@DarshilParmar
Жыл бұрын
Thank you Anurag
@call_of_duty1327
9 ай бұрын
i will add one more service, aws cloudformation for automating all these
@meetpatel.47
4 ай бұрын
Is aws glue and airflow is same?
@chandann7344
Жыл бұрын
Data engineer with AWS Book is best to study and understand AWS
@khalidsk3457
Жыл бұрын
Hi, if u have this book please send me
@chandann7344
Жыл бұрын
I don't have the pdf but i can tell u that orielly's website's providing 10 days free subscription u can read there
@bhaveshnayak6836
Жыл бұрын
If I start learning cloud, which cloud is good and which one can get a job quickly
@Machaivelli
Жыл бұрын
Need your advice brother Between data analyst vs data engineering vs data science 1)Which role requires less coding ? I am a civil engineer who can do decent coding but don’t really enjoyed it like coding geeks do. So need your advice. People suggested me for data analysis profile as it has less doing and needs knowledge of data visualisation but would like to know your inputs 2)which role has more jobs in India ? Data scientist has jobs abroad but in India it’s still in less no and limited to folks from top colleges or brilliant coders, data engineering has decent jobs, no clue about data analyst job scenario Please help
@aka_rony
Жыл бұрын
start with data analysis>after gaining some experience move to DE> then you will get a clear understanding of how an end2end business problem is solved. By that time you will know should I go with data science or stay in Data Engineering (which includes skills of data analyst too). hope this helps, happy learning : )
@bilalgaming123
Жыл бұрын
very informative video and new setup is looking great
@DarshilParmar
Жыл бұрын
Thank you Bilal
@electricalsir
8 ай бұрын
Thanks man
@rakeshverma6867
Жыл бұрын
helpful and providing your insights.
@DarshilParmar
Жыл бұрын
Glad it was helpful!
@Nick-du9ss
Жыл бұрын
Darshil what do you think in future slowly slowly in data engineering cloud services will be used more and decrease open source tools like hive hbase etc.. means data engineering tech stack will go as much as possible on the cloud ?
@DarshilParmar
Жыл бұрын
Open Source will never go away, most of the cloud services use Open Source projects and provide their own feature on top of it, like Managed Airflow, EMR (Hadoop MapReduce) Cloud is future but open is a base for everything
@abhishekravi6975
11 ай бұрын
Great video bro❤
@mohdhammadkhan5570
Жыл бұрын
I used aws glue crawler. I am taking my mssql table to s3 and then through glue to athena so that i can query. In s3 i just checked the columns using objects action button available on bucket and then run a query. So there data is same as mssql. Data in columns where it should be. In athena some columns got mismatched and i don't know how to fix glue crawler now. Kindly help me out Basically crawler got confused because some cols has json type data in rows
@sidsaliyam2725
Жыл бұрын
Man you are a great deal !
@DarshilParmar
Жыл бұрын
Hey thanks man
@bjoe2565
Жыл бұрын
Hello Darshil, Can you please bring some more real time projects? On Azure as well
@DarshilParmar
Жыл бұрын
I am working on it, the thing is creating end-to-end projects takes around 3-4 weeks of efforts and requires lot of planning
@udaybhaskarsavvana8049
Жыл бұрын
Hi Darshil, Need similar one on GCP
@diwakarnrp2092
Жыл бұрын
Hii darshil pls do the project on azure
@DarshilParmar
Жыл бұрын
I will put this on my list
@vaibhavverma1340
Жыл бұрын
Can I put on my resume the real time projects after completing your course ?
@DarshilParmar
Жыл бұрын
Yes you can
@pratikmahajan6082
Жыл бұрын
What about gcp project darshil when he coming
@DarshilParmar
Жыл бұрын
Working on it, will update once it is done
@akashshaw9862
Жыл бұрын
Can't we do all these in Snowflake ? Why would anyone use these services?
@DarshilParmar
Жыл бұрын
Because you can't do everything on snowflake, theoretically you can say (for marketing purpose) but real-world problems are not that straight forward you need custom solution based on requirements
@venkatexplorermetturdam
11 ай бұрын
bro is this sufficient any free resources and shall i do cloud practisner certification im an aspiring data engineer , can u guide please
@DarshilParmar
11 ай бұрын
This is enough but you can do AWS Data Analytics Certification
@venkatexplorermetturdam
11 ай бұрын
@@DarshilParmar thanks bro
@surajitpradhan7848
11 ай бұрын
@@DarshilParmarfrom where we can do it ???
@s.211
Жыл бұрын
Bro bura mat maano but , I am following you since 1 year.. And aap ne last 1 saal main 10 baar roadmap pe video banayi aur wahi baatein repeat ki har video main.. wahi baatein pipeline creation main batate ho... Aap isse accha AWS ki pipelines wali ek ek service in depth padhao
@DarshilParmar
Жыл бұрын
Last 1 year me 5 end-to-end projects bhi banai hai jisme sab service ka indepth explanation and hands-on bataya hai vo bhi dekhlo ;)
@s.211
Жыл бұрын
@@DarshilParmar ok bro , thanks
@endangeredcreator5050
Жыл бұрын
You got 6 more hours to make a aprils fools community post 😂
Пікірлер: 74