Awesome paper, top notch walk-through. Looking forward to more work in interpretability.
@SheikhEddy
9 ай бұрын
I read the post when it came out so didn’t watch this in full, but my main takeaway from this was that since anti-induction heads exist, it’s not necessarily true that a model running on arbitrarily long sequences will always end up repeating itself over and over again.
@BryanWhys
6 ай бұрын
I think it's an emergent phenomenon?
@BryanWhys
6 ай бұрын
I'm pretty sure this is a similar task of the prefrontal cortex
@BryanWhys
6 ай бұрын
Isn't that basically one of the jobs of the prefrontal cortex? Is this emergent or intentionally calculated? It sounds like you're saying it's emergent, and you've discovered that it's happening
Пікірлер: 6