Every LLM provider has its own message shape, token counting, and errors. Here's how to hide all of it behind one Chatter interface, using mixin composition instead of factory boilerplate.
I benchmarked a tiered LLM safety gate with 800 real API calls: a fast model on the hot path, a larger one for the cases that need reasoning, plus circuit breakers and a caution-biased fallback. The measurements upended my reasons for tiering. On current models the fast and large tiers run at nearly the same speed, so the real payoffs are cost and, more importantly, catching the emergencies the cheap model quietly gets wrong.