Thank you for watching, if you enjoyed this episode, please hit like 👍subscribe, and turn notifications on 🔔it helps us more than you know. 🙏
@carlnascnyc
5 ай бұрын
this is indeed a great series, the best one of all youtube imo, it's direct to the point and straightforward, just the right amount of complexity, can't wait for the next chapters (especially the gold zone design/loading), please keep up the good work and many thanks!!! cheers!!
@endjin
4 ай бұрын
Part 8 - Good Notebook Development Practices - is now available: kzitem.info/news/bejne/tq-JnI6LoJpeipg
@clyderodrigo9183
8 ай бұрын
Great series of vids! Keep up the good work. Thank you! Awaiting the next video :)
@endjin
4 ай бұрын
Part 8 - Good Notebook Development Practices - is now available: kzitem.info/news/bejne/tq-JnI6LoJpeipg
@anders1717
6 ай бұрын
Hi finally a no frills explanation of an end to end solution in Fabric thank you! Can you also share the architectural diagram (Visio) you showed us in an earlier video? I'm also looking forward to the next video :)
@endjin
4 ай бұрын
Part 8 - Good Notebook Development Practices - is now available: kzitem.info/news/bejne/tq-JnI6LoJpeipg
@richprice5434
6 ай бұрын
Brilliant video perfect for what I need to do subscribed 🎉
@endjin
6 ай бұрын
That's great to hear! There are a few more videos dropping soon.
@endjin
4 ай бұрын
Part 8 - Good Notebook Development Practices - is now available: kzitem.info/news/bejne/tq-JnI6LoJpeipg
@Tony-cc8ci
6 ай бұрын
Hi, have enjoyed the Microsoft Fabric playlist so far, very informative. This one was 2 months ago, so just wanted to find out if you plan on continuing the series like you mentioned with the helper notebooks?
@endjin
6 ай бұрын
Yes, very much so! Ed (and Barry) are just heads down busy preparing for their workshops and talks at the SQLBits conference next week. As soon as that's done they'll have more capacity to finish off this series of videos.
@endjin
4 ай бұрын
Part 8 - Good Notebook Development Practices - is now available: kzitem.info/news/bejne/tq-JnI6LoJpeipg
@john_britto_10x
4 ай бұрын
In your architecture diagram, you have mentioned HPA orchestrator pipeline. Is that a separate pipeline that needs to be created than the ones at the process layers below?
@endjin
4 ай бұрын
Thanks for the comment! Yes, the orchestrator pipeline is a separate pipeline that includes the configuration that controls the flow of the pipeline. I'll be showing that in an upcoming video, so stay tuned!
@john_britto_10x
4 ай бұрын
@@endjinawesome. Awaiting for it.
@rh.m6660
4 ай бұрын
Not bad, not bad at all. Could you point to a best practice framework for this type of work? Or is this just the way you like to work? I'm finding i can do to the data what i want but am not sure if im being efficient and scalable. I like the idea of using helpernotebooks, i believe cusom spark environments also support custom library's now.
@endjin
4 ай бұрын
This is the way we like to work based on a background of software engineering within the company. We believe data solutions should get the same treatment as software solutions, and for that reason we're huge proponents of applying DataOps practices when building data products. In fact, we've presented at Big Data LDN and SQLBits on this very topic: endjin.com/news/sqlbits-2024-workshop-dataops-how-to-deliver-data-faster-and-better-with-microsoft-cloud Sadly we don't have a specific recording or blog we can share with you (yet!), but part of the point of this series is to expose you to some of our guidance in this area. But stay tuned! Check out "Part 8 - Good Notebook Development Practices" which is now available: kzitem.info/news/bejne/tq-JnI6LoJpeipg --- And you're right that Environments in Fabric support custom libraries. This is another way you can package up your common code and more easily use it across workloads. But this increases the CICD complexity a little bit, so take that into account! Ed
@gabrielepagani763
8 ай бұрын
Hi Ed, thank you for the great and very useful content ! If it's possible... Could you also share the code of the notebooks ?
@endjin
4 ай бұрын
I believe the plan is to release some assets once he's finished the series.
@endjin
4 ай бұрын
Part 8 - Good Notebook Development Practices - is now available: kzitem.info/news/bejne/tq-JnI6LoJpeipg
@MdRahman-wl6qi
Ай бұрын
how to move data from one lakehouse's to another lakehouse table using pyspark?
@Ben-p4p
Ай бұрын
Connect to both lakehouses on the left pane of the notebook. Use spark sql to select and manipulate the data from your bronzelake, then essentially do the merge similar to what is done in the video while selecting the silverlake.
@endjin
27 күн бұрын
Further to Ben's response, I would recommend using two-part naming (.) when transferring this data. However, if you're just wanting to expose the data from one lakehouse in another lakehouse, remember you can avoid copying data altogether by using Shortcuts. And if you need to do this programmatically, there's an API for this: learn.microsoft.com/en-us/rest/api/fabric/core/onelake-shortcuts/create-shortcut?tabs=HTTP#create-shortcut-one-lake-target-example Ed
@john_britto_10x
4 ай бұрын
endjin have you published the 8th part video, if so the link please?
@endjin
4 ай бұрын
It's coming in the next week or so. We've just started a new series you might be interested in kzitem.info/news/bejne/1peIy4R-nqOnhIo
@john_britto_10x
4 ай бұрын
Thank you so much for replying quickly.
@endjin
4 ай бұрын
Part 8 - Good Notebook Development Practices - is now available: kzitem.info/news/bejne/tq-JnI6LoJpeipg
Пікірлер: 28