GitHub shares LLM evaluation practices for production

GitHub's blog details how to evaluate LLMs before production, using its secret scanning system as a case study. It emphasizes starting with the product decision, not the model, and notes that benchmarks may not reflect production distribution.