Topics

Duration

32 Weeks

Progress

0/20

Learning Progress0%

AI Roadmap

Transformers

Learning Path

Follow these concepts in order to master this topic.

Why Transformers?

Limitations of RNNs & LSTMs

Attention Mechanism Review

Self-Attention

Scaled Dot-Product Attention

Multi-Head Attention

Positional Encoding

Encoder Architecture

Decoder Architecture

Encoder-Decoder Models

Masked Attention

Transformer Training

BERT

GPT

Vision Transformers (ViTs)

Fine-Tuning Transformers

Transfer Learning

Prompt Engineering Basics

Real-World Transformer Applications

Documentation

Official documentation and references.

Hugging Face Course

Recommended

Hugging Face Transformers Documentation

Attention Is All You Need

Video Courses

Transformers Explained

27m

Attention Mechanism Explained

26m

Practice Websites

Practice what you've learned.

Hugging Face

Kaggle

Google Colab

Projects

Apply what you've learned by building real projects.

Fine-Tune BERT for Sentiment Analysis

Beginner

Question Answering using DistilBERT

Intermediate

Build a Transformer-based Text Summarizer

Advanced