Model Serving

A bigger foundation model is often the easiest way to get strong general behavior. A small language model can be cheaper, faster, easier to deploy privately, an…

Shrinking a model can mean two very different things. Quantization keeps the model architecture mostly the same but stores and computes weights with lower preci…