AnalysisDevelopers
Jun 16, 6:57 PM
Inference cost at scale with napkin math
Article provides back-of-envelope calculations for estimating LLM inference costs at scale, covering compute and memory requirements based on model size and throughput. Includes practical formulas and examples for capacity planning.