Hugging Face Revamps Kernels for Faster Model Training

Hugging Face has overhauled its kernel system, introducing new APIs and optimizations that can speed up model training by up to 2x on certain hardware. The update includes support for Flash Attention and other memory-efficient algorithms. Developers can now access these kernels through the Transformers library with minimal code changes.