MLP-Mixer: An all-MLP Architecture for Vision
► Paper Abstract:
Convolutional Neural Networks (CNNs) are the go-to model for computer vision. Recently, attention-based networks, such as the Vision Transformer, have also become popular. In this paper we show that while convolutions and attention are both sufficient for good performance, neither of them are necessary. We present MLP-Mixer, an architecture based exclusively on multi-layer perceptrons (MLPs). MLP-Mixer contains two types of layers: one with MLPs applied independently to image patches (i.e. "mixing" the per-location features), and one with MLPs applied across patches (i.e. "mixing" spatial information). When trained on large datasets, or with modern regularization schemes, MLP-Mixer attains competitive scores on image classification benchmarks, with pre-training and inference cost comparable to state-of-the-art models. We hope that these results spark further research beyond the realms of well established CNNs and Transformers.
Paper Link: arxiv.org/abs/...
►Video Outline:
0:00 - Intro
0:42 - Deep Learning on Images
1:47- MLP Architecture
3:51 - Experiments
4:58 - Main Results
5:50 - Results (model scale)
6:17 - Results (invariance to input permutations)
►AI Bites
KZitem: / aibites
Twitter: / ai_bites
Patreon: / ai_bites
Github: github.com/ai-...
►Related Videos:
Swin Transformers: • Swin Transformer: Hier...
Vision Transformers (ViT): • Vision Transformer (Vi...
Data Efficient Image Transformer (DeiT): • DeiT - Data-efficient ...
📚 📚 📚 BOOKS I HAVE READ, REFER AND RECOMMEND 📚 📚 📚
📖 Deep Learning by Ian Goodfellow - amzn.to/3Wnyixv
📙 Pattern Recognition and Machine Learning by Christopher M. Bishop - amzn.to/3ZVnQQA
📗 Machine Learning: A Probabilistic Perspective by Kevin Murphy - amzn.to/3kAqThb
📘 Multiple View Geometry in Computer Vision by R Hartley and A Zisserman - amzn.to/3XKVOWi
Music: www.bensound.com
#deeplearning #machinelearning #aibites
Негізгі бет MLP-Mixer: An all-MLP Architecture for Vision
No video
Пікірлер: 4