LLM evaluation guidelines
Parts, in publishing order
- Evaluation guidelines for LLM applications, part 1: the evaluation checklist
A product evaluation checklist and the business and user expectations to settle before integrating a third-party LLM into an application.
- Evaluation guidelines for LLM applications, part 2: types of evaluation and considerations
The difference between model evaluations and product evaluations, plus coverage, monitoring, user feedback and when fine-tuning is needed.