Henry AI Labs - Organized Video List

Thanks for checking out the Henry AI Labs Video List on Github!
I made this list to group videos together by similar topics such as GANs or Reinforcement Learning.
I think that this is a better way to organize topics in Deep Learning vs. YouTube playlists.

I hope that the categorization of these videos can be additionally useful for those looking to get a general sense of research topics in Deep Learning and AI!

(Some videos are repeated twice if they fall into multiple categories such as Weight Agnostic Neural Networks labeled as "Surprising Findings about Neural Networks" and "Neural Architecture Search")

AI 2019 Recap

This is my recap of Artificial Intelligence in 2019! This video covers developments in the way we understand Neural Networks such as "The Lottery Ticket Hypothesis" and "Weight Agnostic Neural Networks", as well as evolutions in self-supervised learning, language models, generative models, game-playing reinforcement learning, and many more!

https://youtu.be/6SWpN64Ivb4

Understanding Neural Networks

These videos are described as "Understanding Neural Networks" because the reality of Deep Learning is that Neural Networks are far from completely understood. One example of this is "Deep Double Descent" which describes cases where training with large models or datasets actually increases error. This is contrary to the conventional wisdom of training Neural Nets that larger models are Always better. Other papers on this list such as "What's Hidden in a Randomly Weighted Neural Network?" and "The Lottery Ticket Hypothesis" present similar discoveries!

---

Deep Double Descent

This video explores a new study on double descent evident in Deep Learning models such as CNNs, ResNets and Transformers. The double descent phenomenon is an interesting intermediate zone where test error increases and then decreases with respect to increasing the model capacity, adding more training data, and training for longer. This is an interesting phenomenon to be aware of when training Deep Learning models!

https://www.youtube.com/watch?v=R29awq6jvUw&t=1s

What's Hidden in a Randomly Weighted Neural Network: https://www.youtube.com/watch?v=C6Tj8anJO-Q
Randomly Wired Neural Networks: https://www.youtube.com/watch?v=960WIdYaKMM
Weight Agnostic Neural Networks: https://www.youtube.com/watch?v=QqoKl9N2oCw
The Lottery Ticket Hypothesis: https://www.youtube.com/watch?v=LXm_6eq0Cs4

Generative Modeling

One of the most exciting areas of Deep Learning is generative modeling with high-dimensional data like Images, Audio, Text, and Videos. Rather than learning how to classify data points in a high-dimensioanl distribution, generative models try to exactly model the distribution. This is described as modeling p(x) rather than p(y|x) in supervised learning. The videos below describe successful examples of Generative modeling including GANs, VAEs, and Auto-Regressive models (mostly Text).

---

GameGAN: https://www.youtube.com/watch?v=H8F6J7mYyz0
Generative Teaching Networks: https://www.youtube.com/watch?v=lmnJfLjDVrI&t=1s
StyleGANv2: https://www.youtube.com/watch?v=u8qPvzk0AfY&t=11s
SinGAN Explained! (ICCV '19 Best Paper): https://www.youtube.com/watch?v=-f8sz8AExdc&t=4s
VAE-GAN: https://www.youtube.com/watch?v=yyqfZfnSzcw
WaveGAN: https://www.youtube.com/watch?v=BA-Z0KJIyJs
Self-Supervised GANs: https://www.youtube.com/watch?v=-oJWFcexolY
GANs with Fewer Labels: https://www.youtube.com/watch?v=4VqV31TCsZk
GauGAN: https://www.youtube.com/watch?v=dOjES0yJx7s
Self-Attention GAN: https://www.youtube.com/watch?v=OVeGatovZ7Y&t=1s
BigGANs in Data Augmentation: https://www.youtube.com/watch?v=eu7LUwbRyrk
DCGAN: https://www.youtube.com/watch?v=EYrt7fGyA08
SimGAN: https://www.youtube.com/watch?v=ZFwcJfnvTTM
Transfer Learning in GANs: https://www.youtube.com/watch?v=LzD-czp7lcs
Must-Read Papers on GANs: https://www.youtube.com/watch?v=ehDrG98ZgPQ
Improved Techniques for Training GANs: https://www.youtube.com/watch?v=bThj0t703v4
Progressive Growing of GANs: https://www.youtube.com/watch?v=t640zZzIRBY
StyleGAN: https://www.youtube.com/watch?v=AQBti_wN414
StackGAN: https://www.youtube.com/watch?v=s7OIHukdD0o

Neural Architecture Search

Hierarchical Neural Architecture Search: https://www.youtube.com/watch?v=svOpLZ4Zx4A
Weight Agnostic Neural Networks: https://www.youtube.com/watch?v=QqoKl9N2oCw
Evolution in Neural Architecture Search: https://www.youtube.com/watch?v=y0UvVB8k9rI
Wide ResNet: https://www.youtube.com/watch?v=Qyds_i-z0e8
AutoML with Hyperband: https://www.youtube.com/watch?v=eqokKei1aEo&t=359s
SqueezeNet: https://www.youtube.com/watch?v=ge_RT5wvHvY
EfficientNet: https://www.youtube.com/watch?v=3svIm5UC94I&t=40s
Neural Architecture Search: https://www.youtube.com/watch?v=tfCA8X4jGjk&t=4s
ResNet: https://www.youtube.com/watch?v=sAzL4XMke80
Inception Network: https://www.youtube.com/watch?v=n5VQaJc1b14
DenseNets: https://www.youtube.com/watch?v=_8zx4T1Wcmg

Reinforcement Learning

Upside-Down Reinforcement Learning: https://www.youtube.com/watch?v=ed7QQMG24MM
Richard Sutton and Andrew Barto: Introduction to Reinforcement Learning
Chapter 1 Introduction: https://www.youtube.com/watch?v=4SLGEq_HZxk&t=56s
Chapter 2 Multi-Armed Bandits: https://www.youtube.com/watch?v=9LhNHK1ULxs&t=11s
Chapter 3 Finite Markov Decision Process: https://www.youtube.com/watch?v=U24wlvcxXBg&t=99s
Chapter 4 Dynamic Programming: https://www.youtube.com/watch?v=pcZFjPHO4c0&t=269s
Chapter 5 Monte Carlo Methods: https://www.youtube.com/watch?v=uiPhlFrwcw8&t=510s
Chapter 6 Temporal Difference Learning: https://www.youtube.com/watch?v=L64E_NTZJ_0&t=3s
Chapter 7 n-step Bootstrapping: https://www.youtube.com/watch?v=1i5a4yj0Mwg&t=310s
Chapter 8: Planning and Learning: https://www.youtube.com/watch?v=uja8sxJbplg&t=113s

Game-Playing AI

AlphaGo: https://www.youtube.com/watch?v=jgAj8CqcBBs&t=99s
AlphaGo Zero: https://www.youtube.com/watch?v=B1MUfP1qqLs&t=94s
AlphaZero: https://www.youtube.com/watch?v=4FdiTTZPkos&t=93s
MuZero: https://youtu.be/szbvm8aNDxw
The Evolution of AlphaGo to MuZero: https://www.youtube.com/watch?v=A0HX8BgckFI

Coding Tutorials

Vertical Jump Test with Computer Vision: https://www.youtube.com/watch?v=oIqWhCNHa30&t=1s
t-SNE with RAPIDS (600x Speedup): https://www.youtube.com/watch?v=_4OehmMYr44&t=3s
RAPIDS Feature Engineering for the NFL Data Bowl: https://www.youtube.com/watch?v=A9lgUwA8RrY
Slam Dunk Video Classification Tutorial (w/ TF 2.0 Distributed Training!): https://www.youtube.com/watch?v=LQ8b7piC1M4&t=59s
ResNet Keras Implementation: https://www.youtube.com/watch?v=DWpijIMpiPY

Transformers

Towards Human-like Open-Domain Chatbots: https://www.youtube.com/watch?v=STrrlLG15OY
BERT Explained: https://www.youtube.com/watch?v=OR0wfP2FD3c
The Evolved Transformer: https://www.youtube.com/watch?v=khA-fiC1Wa0&t=22s
Reformer: The Efficient Transformer: https://www.youtube.com/watch?v=Kf3x3lqf9cQ&t=382s
CheckList: https://www.youtube.com/watch?v=L3gaWctPg6E&t=7s

Data Augmentation

Adversarial Propagation: https://www.youtube.com/watch?v=KTCztkNJm50&t=77s
RandAugment: https://www.youtube.com/watch?v=Zzt9i3gDueE
Solving Rubik's Cube with a Robot Hand: https://www.youtube.com/watch?v=2AqGocPOOG4&t=2s
Population Based Augmentation: https://www.youtube.com/watch?v=pEANQ8uau88
AutoAugment: https://www.youtube.com/watch?v=2mNP1iMz7mk

Self-Supervised Learning

Self-Training with Noisy Student: https://www.youtube.com/watch?v=Y8YaU9mv_us&t=2s
Semi-Weak Supervised Learning: https://www.youtube.com/watch?v=5cySIwg49RI&t=3s
Self-Supervised GANs: https://www.youtube.com/watch?v=-oJWFcexolY
Multi-Task Self-Supervised Learning: https://www.youtube.com/watch?v=ODG60cYK7aU
Self-Supervised Learning: https://www.youtube.com/watch?v=lbKg3OSTsgA
Unsupervised Data Augmentation: https://www.youtube.com/watch?v=-u8Mi57BDIY
Image Data Augmentation for Deep Learning: https://www.youtube.com/watch?v=mljRx81K1gY
BigGANs in Data Augmentation: https://www.youtube.com/watch?v=eu7LUwbRyrk
Unsupervised Feature Learning: https://www.youtube.com/watch?v=YniWmMxlKvY

Meta Learning

Generative Teaching Networks: https://www.youtube.com/watch?v=lmnJfLjDVrI&t=1s
Solving Rubik's Cube with a Robot Hand: https://www.youtube.com/watch?v=2AqGocPOOG4&t=2s

Working with Neural Networks

Mixed Precision Training: https://www.youtube.com/watch?v=pKZs4hllCvI&t=520s
Adversarial Propagation (Vision Models): https://www.youtube.com/watch?v=KTCztkNJm50&t=77s
Inference in Deep Learning: https://www.youtube.com/watch?v=kPQQ61Ks46A&t=2s
The ReLU Activation Function: https://www.youtube.com/watch?v=Ei6274NnDvU

Curriculum Learning

Learning to Execute: https://www.youtube.com/watch?v=5rrldwJdDRE
Teacher-Student Curriculum Learning: https://www.youtube.com/watch?v=GFCujBpTf3k
Curriculum Learning in Deep Neural Networks: https://www.youtube.com/watch?v=hV61aU3UL-w

Semi-Supervised Learning

FixMatch: https://www.youtube.com/watch?v=nkewn6XGyt8&t=1s
Semi-Weak Supervised Learning: https://www.youtube.com/watch?v=5cySIwg49RI&t=3s

Knowledge Distillation

Knowledge Distillation with TAs: https://www.youtube.com/watch?v=ueUAtFLtukM&t=218s
Self-Training with Noisy Student: https://www.youtube.com/watch?v=Y8YaU9mv_us&t=2s

Deep Learning on Graphs

Graph Convolutional Networks: https://www.youtube.com/watch?v=pH1Zz6c1Q7A
Embedding Graphs with Deep Learning: https://www.youtube.com/watch?v=tfyxxGXCpAg
DeepWalk: https://www.youtube.com/watch?v=N_XOTh3SDZ8

Weakly Supervised Learning

Semi-Weak Supervised Learning: https://www.youtube.com/watch?v=5cySIwg49RI&t=3s
Weakly Supervised Pretraining: https://www.youtube.com/watch?v=m2ofSCpRFGk

Multi-Task Learning

Multi-Task Self-Supervised Learning: https://www.youtube.com/watch?v=ODG60cYK7aU

Neuroevolution

Coevolution of Agents and Environments (POET): https://www.youtube.com/watch?v=YBC-2zccO0s&t=5s
Novelty Search for Neuroevolution: https://www.youtube.com/watch?v=-mxpn95uxS4
Genetic CNN: https://www.youtube.com/watch?v=GZMcy_vl5wA
CoDeepNEAT: https://www.youtube.com/watch?v=XvCbgwhMVu4
Neuroevolution of Augmenting Topologies (NEAT): https://www.youtube.com/watch?v=b3D8jPmcw-g&t=54s
Weight Agnostic Neural Networks: https://www.youtube.com/watch?v=QqoKl9N2oCw
Evolution in Neural Architecture Search: https://www.youtube.com/watch?v=y0UvVB8k9rI
Population Based Training: https://www.youtube.com/watch?v=pEANQ8uau88

Few-Shot Learning

Siamese Neural Networks: https://www.youtube.com/watch?v=T9yKyZfxUJg

Attention

Show, Attend and Tell: https://www.youtube.com/watch?v=bBMxSg3c_6M
Self-Attention GAN: https://www.youtube.com/watch?v=OVeGatovZ7Y&t=1s

Efficient Deep Learning

RevNet: Backpropagation without Storing Activations: https://www.youtube.com/watch?v=EulWJgvNWfM

Deep Compression

Deep Compression: https://www.youtube.com/watch?v=xDS7ljg0T-E

Applications of Deep Learning

Skin Cancer Classification with Deep Learning: https://www.youtube.com/watch?v=GkTgSgjJuW8
Video Classification with Deep Learning: https://www.youtube.com/watch?v=LAV56E-mWoo

Miscellanious Videos

Google Research at ICCV: https://www.youtube.com/watch?v=z-yvY8iAaHM&t=620s
Facebook Research at ICCV: https://www.youtube.com/watch?v=W5EsADGw9CA&t=30s
Word2Vec: https://www.youtube.com/watch?v=cQFOxMkzwf4

Podcasts

Yannic Kilcher: https://www.youtube.com/watch?v=084W48_uEz0&t=120s
Edward Peake: https://www.youtube.com/watch?v=Z-SePjfKAYM
Edward Dixon: https://www.youtube.com/watch?v=pwaUfkFZTDE

from tensorflow import keras
from tensorflow.keras import layers

vision_model = keras.applications.<MASK>