It is slightly comforting that even "high profile" papers often have meh ideas hold together with duct tape.
@josephharvey1762
2 жыл бұрын
I love the way you freely critique papers!
@bennettbullock9690
2 жыл бұрын
I have to read the article, but the Chomskian (conventional linguistic) perspective is that grammar is an aide to semantic interpretation, and therefore the LMM's knowledge of grammar is going to be definition encode knowledge of at least simple facts. Which leads me to wonder why we even bother with extracting a relation from attention matrices in the first place. Why not just extract them from the sentence itself, since the sentence specifies this relation?
@andres_pq
3 жыл бұрын
Loved the South Park reference 17:30
@sarvagyagupta1744
3 жыл бұрын
I think you should make a video on the differences between these attention techniques and Lambda layers. Similarities and differences can be quite confusing.
@DinHerios
3 жыл бұрын
Click-bait generates more clicks, which leads to more advertising money. Or, as in the case of academia, potentially more cites, simply because more people read the paper. It was only a question of time before academia gets infected with this concept. It certainly seems to work really well.
@G12GilbertProduction
3 жыл бұрын
In SEO sites optimalization attended to search engine indexes it's probably being more copy only this same tags at same time.
@ChaoticNeutralMatt
10 ай бұрын
Yeah. This wasn't what I expected and I'm somewhat disappointed. At least it was fairly immediate that this wasn't what I expected based on the abstract.
@drhilm
3 жыл бұрын
Even your commercials are interesting.
@PeterOtt
3 жыл бұрын
wow, TabNine doesnt even start auto-charging you after the 3 months expire? what a good guy thing to do!
@karnigilon
3 жыл бұрын
As you say much of this seems like "grammatical" discovery. How does it compare to simply using the verbs as a relation? maybe using the verbs as the basis , and then using the attention values to filter out some of the tuples can increase the recall?
@YannicKilcher
3 жыл бұрын
good idea, I guess the goal here was to publish, not to get the best numbers :)
@herp_derpingson
3 жыл бұрын
I think while this might work great in Wikipedia like well written corpuses. It will fail miserably for spoken human speech, as it is very noisy. But I like it. Simple idea but it will take you a long way.
@horizon9863
3 жыл бұрын
I tried TabNine but the CPU usage is really high, every time it starts CPU got 50% usage (AMD 3900X)
@eduarddronnik5155
3 жыл бұрын
Dawn Song is a seriously great name. Like Klara Himmel.
@robertoc.martineza.3719
Жыл бұрын
2 years later: GPT4 can make nice KG using ASCII art or pixel art with specific color palette.
@florianhonicke5448
3 жыл бұрын
yeahhhh, another week - another paper!
@shengyaozhuang3748
3 жыл бұрын
Many nouns will be split into word pieces right? how to compute attention weights for those nouns?
@YannicKilcher
3 жыл бұрын
good question, idk
@TheZork1995
3 жыл бұрын
I read that there is no way in combining the subword parts in a meaningful way. So maybe they just use the Subword embedding to represent the word. So a big word has more chances. But it's just a guess.
@DistortedV12
3 жыл бұрын
Can you go from updated knowledge graph to language model again? If so, that would be really cool
@skm460
3 жыл бұрын
Won't it be an easier task to just form sentences from the triplets?
@raphaels2103
2 жыл бұрын
There is a method, rome
@konghong3885
3 жыл бұрын
I would argue the model in the commercial is more interesting then the results shown in the paper (good explanation tho) KG suffered with so many problems (graph incomplete, graph retrieval problems) making it a nightmare to use in production this result is just again demonstrate the power of transformers to perform in fewshot tasks, even if the task is knowledge compression (KG)
@kikimajo6850
3 жыл бұрын
1⃣️ Knowledge in Corpus: Dylan is a songwriter 2⃣️ "Knowledge" given by spaCy: Dylan(Noun), songwriter(Noun) 3⃣️ "Knowledge" in pre-trained models: The word "Dylan" somehow relates to "is", syntactically or semantically. AND The word "is" somehow relates to "songwriter". 1⃣️2⃣️3⃣️ -----MATCH----> (Dylan, is, songwriter) (Dylan, is, songwriter) -----MAP-----> KG(Bob_Dylan, occupation, songwriter) It seems that 3⃣️ is not 'knowledge' but 1⃣️ actually is.🤔🤔
@editorsworld1521
3 жыл бұрын
Can't agree more, kind of overstating, especially its title...
@soopace1486
3 жыл бұрын
thank you for the interesting video
@sillygoose2241
3 жыл бұрын
The relation must be in between the head and the tail?? Yoda pleased is not
@tinyentropy
2 жыл бұрын
After half of the video, I felt it is a waste of time. Unfortunately, you didn't properly set expectations by the title of the video.
@MachineLearningStreetTalk
3 жыл бұрын
Hello 🙌
@YannicKilcher
3 жыл бұрын
Hello 👋
@maxwellclarke1862
3 жыл бұрын
I don't get the comment about Rust :) ?
@evilunicorn9205
3 жыл бұрын
Dawn Song is a badass name
@first-thoughtgiver-of-will2456
3 жыл бұрын
is regressing on human input knowledge distillation?
@YannicKilcher
3 жыл бұрын
good point
@SirPlotsalot
Жыл бұрын
There's a case to argue it's actually an example of optimally-robust curriculum learning which is neat
@kavitachoudhary1112
3 жыл бұрын
superb video .please guide or suggest new area of NLP from where i start and go for research.,thank u in advance please guide
@YannicKilcher
3 жыл бұрын
rare languages are a hot topic
@DamianReloaded
3 жыл бұрын
EDIT: Can't gpt3 do this already? Like giving it a text as prompt and get it to generate an xml containing the triples in the text you gave it? O_o
@ea_naseer
Жыл бұрын
I'm here from the future, gpt3 can generate PDDL statements fine so it can probably generate triples in text
@marouanemaachou7875
3 жыл бұрын
Keep it coming ! haha
@manojb8876
3 жыл бұрын
Wait, is tab9 even legal. Isn't gpt not supposed to be released to companies and now bought by Microsoft?
@BryanRink
3 жыл бұрын
It's based on GPT2, which is released. You're thinking of GPT3.
@G12GilbertProduction
3 жыл бұрын
I think this tetradecimal language structure of this model was not really tetradecimal than be are, only sextadecimal.
@first-thoughtgiver-of-will2456
3 жыл бұрын
dont bait me with the rust shoutout.
@interestingamerican3100
Жыл бұрын
What is wrong with Rust?
@dr.mikeybee
3 жыл бұрын
Thanks for the free 100 days.
@7th_CAV_Trooper
Жыл бұрын
Thumbs up for South Park reference.
@bobsavage3317
Жыл бұрын
This paper makes too many assumptions. For example, "Is" and "Was" are looked up as different Relations. Also, "To Be" is often a sign that the terminal Entity is actually a property of the Subject leading to semantic statements like P(x, TRUE), e.g. the triple (Ernst, Pacifist, TRUE). Another apparent assumption is that a sentence will only have 1 fact in it. The list goes on! It's a shame, because the title suggests there could be a way to extract the semantic concept in the model and externalize them in a manner that is machine verifiable. The ability to audit a model on a given topic would be very helpful.
@ChaoticNeutralMatt
10 ай бұрын
This feels very unexplored and unrefined. I'll have to hold onto this.
Пікірлер: 63