I've covered these before I got the news that I'll be joining DeepMind. 😅 And in any case I make these videos independently from the company I work at. I want to cover the most exciting research and engineering ideas.
@minos99
2 жыл бұрын
Wait...what? DeepMind! Ooh...we're gonna be be getting bleeding edge leaks😅😅😅. Congrats. 👏👏👏I'm not too surprised. And don't stop what you do here...
@TheAIEpiphany
2 жыл бұрын
@@minos99 hahaha thanks! If I start leaking you can soon expect ex-DeepMind. 😅😂
@48956l
2 жыл бұрын
it would take quite a shallow mind to leak your company's research lol but congrats!
@minos99
3 жыл бұрын
Awesome coverage. Kinda reminds me of end to end memory network.
@DistortedV12
2 жыл бұрын
Was waiting on this one and CONGRATS ON DEEPMIND!!
@TheAIEpiphany
2 жыл бұрын
Hahah thanks! 😄🚀
@hillolmaity2631
3 жыл бұрын
Thank you so much for covering this
@patf9770
2 жыл бұрын
Although the big contributions are efficiency, generality, and the ability to handle multiple modalities; Perceiver seems very useful to change between arbitrary and fixed size sets/sequences. One immediate use case is to get better sentence/paragraph embeddings.
@jawadmansoor6064
2 жыл бұрын
Thank you for explaining. I think your explanations are better than others. I generally understand you easily. Keep doing the good work.
@TheAIEpiphany
2 жыл бұрын
Thank you man - appreciate it! It's only going to get better I've still got lots of room for improvement.
@jawadmansoor6064
2 жыл бұрын
@@TheAIEpiphany The only thing that can improve (though not required right now) is animation. If you could show where to tokens go and how do they transform (in every video). That would make things even easier to understand. I will subscribe to your discord (got some queries, no pun intended. XD)
@TheAIEpiphany
2 жыл бұрын
@@jawadmansoor6064 Of course but it takes too much time and then I can output less content - that's the tradeoff. 😅 Thanks for flagging though.
@jawadmansoor6064
2 жыл бұрын
@@TheAIEpiphany Understandable. That is why I think for now easily understandable content has more value (you do more research work than work hard on animation). Thank you for being nice and producing quality content.
@hadjdaoudmomo9534
2 жыл бұрын
Excellent explanation! Thank you
@aleksey3231
2 жыл бұрын
Omg, why music started at 30:00?))
@PaganPegasus
2 жыл бұрын
I guess it would be permutation invariant instead of equivariant, since we're referring to the permutation of the K/V byte array input but constant latent Q array input: the output of the attention layer will be identical regardless of the order of the K/V input so this is permutation invariant.
@444haluk
2 жыл бұрын
27:13 tshirt changed :P
@stephennfernandes
3 жыл бұрын
Thanks a lot for Covering this 🔥🔥 .. Congratulations on making it to DeepMind
@TheAIEpiphany
3 жыл бұрын
You're welcome! Thanks man!
@stephennfernandes
3 жыл бұрын
@@TheAIEpiphany hey if you don't mind, we all would love to know about how your interview process was in DeepMind, what were the question's asked, what kind of work you need to put into making it into DeepMind ?, I know you haven't been posting anything non technical on this channel, and i totally respect your taste. But it would really mean the world to a young aspirant like me who dreams of making it to DeepMind some day. If possible please make a video about it
@TheAIEpiphany
3 жыл бұрын
@@stephennfernandes Don't worry I'll be making that content!
@TheAIEpiphany
3 жыл бұрын
@@stephennfernandes if you take a look at some of my older videos those were not all research papers and coding videos. 😄
@stephennfernandes
3 жыл бұрын
@@TheAIEpiphany thank a lot man, anxiously awaiting on that 😍
@alexanderchernyavskiy9538
2 жыл бұрын
outrageous explanation; hope this could help with the probing tasks and multimodality tasks along with conv mixers;)
@pratik245
2 жыл бұрын
What original ideas, these guys are alien smart
@TheAIEpiphany
2 жыл бұрын
Well, not quite. Many of the ideas were already floating around. 😄 It depends on the context you got how you'll perceive a result like Perceiver (no pun intended).
@robboswell3943
Жыл бұрын
Excellent video! A question for you: How exactly are the learned latent arrays being learned? Perhaps I'm wrong, but it would seem that there may be a special algorithm used to create the learned latent array that somehow reduces the dimensions from the input "byte array"? For example, on pg. 3 of the Perceiver paper it states, "The model can also be seen as performing a fully end-to-end clustering of the inputs with latent positions as cluster centres..." Do the authors use an unsupervised learning clustering algorithm of some kind to do this?
@tonibagur3238
2 жыл бұрын
Hi, it seems that in Perceiver the input is iteratively mixed with the latents via cross attention but in Perceiver IO it seems that the input is feed only once in the network. Does somebody know if I am right and the reason of doing it in this way?
@alvarofrancescbudriafernan2005
2 жыл бұрын
What is the first, initial latent array? How do you choose it? Byte array is the usual input (e.g. image) but what is laten array for?
@yassineabbahaddou4369
2 жыл бұрын
why they have used a GPT architecture instead of BERT for the latent attention ?
@felipe7x
2 жыл бұрын
What applications this could have for marketing or retail? Thanks
Пікірлер: 33