Optimizing Jagged Flash Attention with TLX: The Road Toward SOTA FA4 on Blackwell

TL;DR In this blog post, we present our work on Jagged Flash Attention (JFA) — the attention kernel behind Meta’s Generative Ads Model (GEM) — on NVIDIA Blackwell (B200), built...

Analyze with AI

By Han Xu, Jacky Zhou, Jackie (Jiaqi) Xu, Hongtao Yu, Peng Chen (Dev Infra), Darren Liu, Dev (Devashish) Shankar, Max Leung, Nick Riasanovsky, Hao Yan, Manman Ren, Yuanwei (Kevin) FangPyTorch Blog

Optimizing Jagged Flash Attention with TLX: The Road Toward SOTA FA4 on Blackwell

Read the full story

Originally published by PyTorch Blog on Oct 1, 2026.

Read on PyTorch Blog

Comments

No comments yet — be the first to share your thoughts.