Tag: Local AI

A laptop, a single GH200, and a semantic router between them, built to answer trivial requests locally and send the hard ones to the big card. Along the way, eleven things in that stack told me a confident falsehood. Not one of them was the model, and three of them were my own measurements.

I spent a night carefully A/B testing a model on a GH200. Then I upgraded the inference engine for an unrelated reason, re-ran the identical config, and got 43 percent more decode throughput and nearly four times the context budget. When open-weight architectures change this fast, the engine is the other half of the model, and it fails quietly.