Note
Pre-Layer Normalization in Transformers
A short note on why moving LayerNorm before attention and feed-forward blocks can improve training stability.
Idea
Post-LN applies normalization after the residual branch. Pre-LN applies it before the attention or feed-forward sublayer.
Why it matters
Pre-LN often makes gradient flow easier in deeper Transformer stacks because the residual path remains more direct.