Machine Learning - Deep Learning Overview

Minimal training loop (PyTorch)

for epoch in range(E):
    model.train(); total = 0
    for X, y in loader:
        opt.zero_grad(); loss = loss_fn(model(X), y); loss.backward(); opt.step()
        total += loss.item()
    print({"epoch": epoch, "loss": total/len(loader)})

Architectures

• CNNs: vision; locality and shared weights.
• RNNs: sequences; many tasks now use Transformers.
• Transformers: attention; state of the art in NLP/vision/audio.

Details

• Convolutional Neural Networks (CNN)
• Recurrent Neural Networks (RNN)
• Transformers