Video by MLflow via YouTube
In this sixth episode of this series, Jules Damji dives deep into MLflow’s extensive framework integrations. MLflow supports over 30 different open-source agent-building frameworks, allowing you to automatically trace and evaluate complex AI workflows regardless of your chosen architecture.
This tutorial provides a hands-on comparison of three open source agent building frameworks and demonstrates how MLflow provides full visibility into their execution:
🔹 𝗟𝗮𝗻𝗴𝗖𝗵𝗮𝗶𝗻: Learn how to use high-level primitives like ChatPromptTemplate and StringOutputParser to build sequential workflows. We demonstrate both simple chains and complex multi-step sequences connected via the pipeline operator.
🔹 𝗟𝗹𝗮𝗺𝗮𝗜𝗻𝗱𝗲𝘅: See how to build a Retrieval-Augmented Generation (RAG) system. We walk through creating an in-memory vector index, generating embeddings with OpenAI, and using a query engine to retrieve document-based answers, all while capturing the entire operation trace in MLflow.
🔹 𝗟𝗮𝗻𝗴𝗚𝗿𝗮𝗽𝗵: For more advanced use cases, we explore building stateful, hierarchical agent workflows. We demonstrate a customer service triage system that uses a supervisor node to classify queries and route them to specialized handlers for billing, tech support, or general inquiries.
Key Takeaways:
🔹 𝗔𝘂𝘁𝗼𝗺𝗮𝘁𝗶𝗰 𝗧𝗿𝗮𝗰𝗶𝗻𝗴: All frameworks integrated with MLflow are automatically traced, capturing inputs, outputs, and intermediate steps without manual instrumentation.8
🔹 𝗙𝗿𝗮𝗺𝗲𝘄𝗼𝗿𝗸 𝗦𝗲𝗹𝗲𝗰𝘁𝗶𝗼𝗻: Choose LangChain for sequential chains, LlamaIndex for heavy document indexing, and LangGraph for complex, stateful branching or looping workflows.
🔹 𝗩𝗶𝘀𝗶𝗯𝗶𝗹𝗶𝘁𝘆: Use the MLflow UI to inspect timelines, verify embeddings, and debug the internal logic of your AI agents.
Resources:
🔗 Notebook 1.5: https://github.com/dmatrix/mlflow-genai-tutorials/blob/main/06_framework_integrations.ipynb
🎥 Full Series Playlist: https://youtube.com/playlist?list=PLaoPu6xpLk9EI99TuOjSgy-UuDWowJ_mR&si=jdbAbxTCRuxFxfnG