Home AI slashes reply time from 4 minutes to 11 seconds

A developer running a 32B language model locally on an AMD integrated GPU reduced first-reply latency from 242 seconds to 11 seconds. Key optimizations included disabling unnecessary reasoning tokens, removing unused tool schemas, caching prompts per session, and scheduling cold starts. The project highlights practical challenges of running large models on consumer hardware.