Self-speculative decoding speeds AI fine-tuning without quality loss

A new paper introduces self-speculative decoding, which creates a compressed copy of the model at each training step to draft text faster. The full model verifies the drafts, achieving meaningful speedups in generation with no loss in final model quality. The technique is lossless and shaves time off the slowest step of reward-based fine-tuning.