Thanks bro, that is such an elegant and concise explanation of the concept! Really Helpful
@MissPiggyM976
21 күн бұрын
Very good!
@yingjiawan2514
Ай бұрын
This is so well explained. thank you so much!!! Now I know how to understand KL divergence, cross entropy, logits, normalization, and softmax.
@chunheichau7947
Ай бұрын
I wish more professors can hit all the insights that you mentioned in the video.
@Sars78
Ай бұрын
Well done, Adian. I just found out-though I'm not surprised at all, in the Shannon sense 🤓 -that you're doing a PhD at Cambridge. Congratulations! Best wishes for everything 🙂
@user-bi2jm1cn1h
Ай бұрын
How does the use of soft label distributions, instead of one-hot encoding hard labels, impact the choice of loss function in training models? Specifically, can cross-entropy loss still be effectively utilized, or should Kullback-Leibler (KL) divergence be preferred?
@sanjayadhith
2 ай бұрын
Useful video.
@HaykTarkhanyan
2 ай бұрын
great video, thank you!
@AdeshBenipal
2 ай бұрын
Nice video
@Micha-ku2hu
3 ай бұрын
What a great and simple explanation of the topic! Great work 👏
@xyzct
3 ай бұрын
Excellent. Short and sweet.
@genkidama7385
3 ай бұрын
distirbution
@avatar00001
3 ай бұрын
thank you codexchan
@mathy642
3 ай бұрын
Thank you for the best explanation
@franklyvulgar1
4 ай бұрын
this is a great explanation thank you !
@debasishraychawdhuri
4 ай бұрын
It does not explain the most important part - how the formula for non-uniform distribution came about
@shahriarrahman8425
4 ай бұрын
Great explanation. Thank you so much!
@ian-haggerty
4 ай бұрын
Adian. I know you're probably super busy doing PhD things, but come back & make some more videos! You're a gifted orator.
@ian-haggerty
4 ай бұрын
Best explanation on the interwebs!
@madarahuchiha1133
4 ай бұрын
what is true class distribution?
@elenagolovach384
3 ай бұрын
the frequency of occurrence of a particular class depends on the characteristics of the objects
@ian-haggerty
5 ай бұрын
So a Kale Divergence of zero means identical distributions? What do the || lines mean?
@ian-haggerty
5 ай бұрын
<3 this. 👌
@user-ut4zh3pw7l
5 ай бұрын
wowowowo
@adityakulkarni5577
5 ай бұрын
Perfectly explained in 5 minutes. Wow.
@LiHongxuan-ee7qs
5 ай бұрын
So clear explanation! Thanks!
@charleswilliams8368
5 ай бұрын
Three bits to tell the guy on the other side of the wall what happened, and it suddenly made sense. Thanks.
@EricPham-gr8pg
6 ай бұрын
This is an incomplete hypothesis because on p(x) and ln(p(x)) come without proof is grossly insulting and it ignored the basic principle of conservation of energy meaning if pressure goes up then volume also goes up to release presured to go back to normal and that is the frustrating truth of life. The beast never dies it only change its phases or faces
@SunilKumarSamji
6 ай бұрын
Excellent video. Can someone help me understand why is it called Divergence in the first place? Why are we taking 1/N power to normalise it to sample space, I did not understand the logic behind this.
@ajitzote6103
6 ай бұрын
not really a great explaination, so many terms were thrown in. that's not a good way to explain something.
@user-cf2yo5qf3h
6 ай бұрын
Thankkkk youuuuu.
@brianlee4966
6 ай бұрын
Thank you so much for this video and clear explanation!
@jiwoni523
7 ай бұрын
make more videos please , you are awesome
@ananthakrishnank3208
7 ай бұрын
Excellent expositions on KL divergence and Cross Entropy loss within 15 mins! Really intuitive. Thanks for sharing.
@cuongnguyenuc1776
7 ай бұрын
Great video! Can you make a video about soft actor critic?
@sirelegant2002
8 ай бұрын
Thank you!
@zizo-ve8ib
8 ай бұрын
Bro really explained it in less than 10 mins when my professors don't bother even if it could be done in 5 secs, true master piece thus video keep it up man 🔥🔥🔥
@wynandwinterbach455
8 ай бұрын
I'm just rewatching this video to freshen up my deep learning fundamentals. Super clear video, thank you so much!
@bodwiser100
8 ай бұрын
I appreciate your effort, but the video is quite confusing. For example, in the example about 8 football teams, you explain why 3 bits are required by flat out stating as a starting premise that 3 bits are required! It's a circular argument.
@nicholaselliott2484
9 ай бұрын
Dude, took information theory from a rigorously academic and formal professor. I'm a little slow and under the pressure of getting assignments done, couldn't always see the forest for the trees. Just the sentence "how much information, on average, would we need to encode an outcome from a distribution" just summed up the whole motivation and intuition. Thanks!
@nyx8017
9 ай бұрын
god this is an incredible video thank you so much
@aj7_gauss
9 ай бұрын
can someone explain the triplets part
@kukuster
9 ай бұрын
Thanks for the explanation!! One thing is, formulas were confusing with how you denoted *q1* & *q2* for probabilities for coin 2, instead of *p2* & *q2=1-p2*
@mormonteg4073
10 ай бұрын
Thank you a lot
@akidnag
10 ай бұрын
Only that is not a distance ('cause is not symmetric), but a pseudo distance. Great video!
@diy_mo
10 ай бұрын
I expected something else, but it's also ok.
@karthikeyans3
10 ай бұрын
Great video. Thanks for sharing. Really intuitive.
@maxlehtinen4189
10 ай бұрын
for everyone trying to understand this concept even more thoroughly, towardsdatascience's article "The intuition behind Shannon’s Entropy" is amazing. it gives added insight on why information is the reciprocal of probability
@avaodonnell128
7 ай бұрын
Thank you!!
@benjahnz
4 ай бұрын
And it's paywalled...
@bluejays440
10 ай бұрын
Please make more videos this is literally the only time I've ever seen entropy be explained in a way that makes sense
Пікірлер