Run LLMs at Home with BitTorrent-Style Distributed Inference

Petals lets users run large language models like Llama 2 by pooling GPU resources from multiple peers, similar to BitTorrent. The open-source platform splits model layers across participants, enabling inference on consumer hardware. It supports models up to 65B parameters with collaborative computing.