Video by H2O.ai via YouTube
You cannot trust what you do not measure. Evaluation looks different depending on whether you are assessing a model, a retrieval system or an agent.
Terms covered:
– Evaluation: systematically measuring quality, accuracy and reliability. Not a one-time check at the end of development, but an ongoing practice.
– LLM Evaluation: factual accuracy, relevance, coherence, tone and safety, measured across thousands of inputs and edge cases
– RAG Evaluation: assessing retrieval and generation separately, because poor retrieval produces unreliable answers no matter how good the model is
– Agent Evaluation: judging decisions and actions, not just text. Did it complete the task, use the right tools, handle the unexpected?
– Tracing Matrix: a structured framework mapping inputs to outputs, so failure points can be isolated and tracked over time
Evaluation is what turns AI from a promising technology into a trustworthy one.
—
AI Fundamentals Glossary | H2O.ai University
Full playlist: https://www.youtube.com/playlist?list=PLAWKlIKS-66E
Free courses and certifications: https://h2o.ai/university
#AIEvaluation #ResponsibleAI #LLM #AIGlossary #H2Oai