UGR: An AI Runtime OS That Runs LLMs on Commodity Hardware

Developer Sumeet is building UGR, an AI runtime operating system that orchestrates inference across memory tiers, enabling LLMs to run on older GPUs with 4GB VRAM. It sits above existing engines like llama.cpp, simulating execution to predict memory use and throughput. The project aims to decouple AI capability from hardware constraints.