Video by MLflow via YouTube
In this sixth episode of this series, Jules Damji dives deep into MLflow’s extensive framework integrations. MLflow supports over 30 different open-source agent-building frameworks, allowing you to automatically trace and evaluate complex AI workflows regardless of your chosen architecture.
This tutorial provides a hands-on comparison of three open source agent building frameworks and demonstrates how MLflow provides full visibility into their execution:
๐น ๐๐ฎ๐ป๐ด๐๐ต๐ฎ๐ถ๐ป: Learn how to use high-level primitives like ChatPromptTemplate and StringOutputParser to build sequential workflows. We demonstrate both simple chains and complex multi-step sequences connected via the pipeline operator.
๐น ๐๐น๐ฎ๐บ๐ฎ๐๐ป๐ฑ๐ฒ๐
: See how to build a Retrieval-Augmented Generation (RAG) system. We walk through creating an in-memory vector index, generating embeddings with OpenAI, and using a query engine to retrieve document-based answers, all while capturing the entire operation trace in MLflow.
๐น ๐๐ฎ๐ป๐ด๐๐ฟ๐ฎ๐ฝ๐ต: For more advanced use cases, we explore building stateful, hierarchical agent workflows. We demonstrate a customer service triage system that uses a supervisor node to classify queries and route them to specialized handlers for billing, tech support, or general inquiries.
Key Takeaways:
๐น ๐๐๐๐ผ๐บ๐ฎ๐๐ถ๐ฐ ๐ง๐ฟ๐ฎ๐ฐ๐ถ๐ป๐ด: All frameworks integrated with MLflow are automatically traced, capturing inputs, outputs, and intermediate steps without manual instrumentation.8
๐น ๐๐ฟ๐ฎ๐บ๐ฒ๐๐ผ๐ฟ๐ธ ๐ฆ๐ฒ๐น๐ฒ๐ฐ๐๐ถ๐ผ๐ป: Choose LangChain for sequential chains, LlamaIndex for heavy document indexing, and LangGraph for complex, stateful branching or looping workflows.
๐น ๐ฉ๐ถ๐๐ถ๐ฏ๐ถ๐น๐ถ๐๐: Use the MLflow UI to inspect timelines, verify embeddings, and debug the internal logic of your AI agents.
Resources:
๐ Notebook 1.5: https://github.com/dmatrix/mlflow-genai-tutorials/blob/main/06_framework_integrations.ipynb
๐ฅ Full Series Playlist: https://youtube.com/playlist?list=PLaoPu6xpLk9EI99TuOjSgy-UuDWowJ_mR&si=jdbAbxTCRuxFxfnG