Video by Open Data Science and AI Conference via YouTube
In this episode of the ODSC Ai X Podcast, host Dan Gerlanc speaks with Scott Askinosie, Senior Principal AI Engineer at Steel Engine, an AI workspace for building AI agents.
In this conversation, Scott explains why production agent failures are often less about model capability and more about the systems, governance, and workflows around the AI. The episode explores how companies can move from pilots to production-ready agentic systems, why deterministic workflows and observability matter, and why humans and agents will likely work together as part of the same operating system.
0:00 Introduction to Scott Askinosie and the episode
1:05 What breaks most often in production agent systems
3:13 Guardrails for agentic AI systems
3:55 OpenClaw, autonomy, and enterprise risk
5:13 Protecting PII and sensitive data in agent workflows
6:37 Why agents should not have direct access to passwords or keys
7:09 Use the simplest tool possible
8:02 Deterministic code, model routing, and governance
8:46 Why governance helps enterprises adopt agentic AI
9:41 Deterministic pathways, skills, and in-context learning
10:16 Orchestration as a governance layer
11:00 Lessons from OpenClaw failures and agent autonomy
12:38 Steel Engine and governance for enterprise agents
13:00 Using n8n, MCP, Airflow, and Flyte for agent workflows
14:00 Production trade-offs: cost, traceability, memory, and architecture
15:11 How AI accessibility is changing enterprise teams
16:20 From software engineers to vibe coders
17:00 Why internal AI teams are becoming more common
18:46 Build vs. buy for enterprise agentic systems
21:37 Why teams need to keep up with fast-changing AI tools
22:19 Why classical software engineering skills still matter
23:26 Learning with coding agents instead of just vibe coding
24:00 AI engineering vs. agentic engineering
24:57 Why AI will not simply replace all human work
27:29 Inference costs, layoffs, and the future of technical work
28:36 Will companies rehire engineers?
29:15 Why AI works better alongside humans
30:23 Why responsibility cannot be fully delegated to AI
31:00 Companies returning to AI after failed production rollouts
33:21 Observability and evaluations for production agents
34:16 Deterministic pipelines, step logging, and human review
34:36 Aviation use case: automating legacy workflows with agents
36:27 Favorite tracing and evaluation tools
36:49 Comet Opik, Langfuse, Grafana, and Datadog
38:00 Open models vs. commercial models in enterprise AI
39:43 What Scott is watching next in agentic AI
40:00 Mythos, Project Glasswing, and new model capabilities
40:31 Managing humans and agents together
41:00 Multimodal models, Gemini, OpenBrain, and personal AI memory
41:48 Closing thoughts
Visit our website and choose the nearest ODSC AI event to attend and experience all our training and workshops: https://odsc.ai
To watch more videos like this, visit https://aiplus.training
Sign up for the newsletter to stay up to date with the latest trends in data science: https://opendatascience.com/newsletter/
Follow us online!
• Facebook: https://www.facebook.com/OPENDATASCI
• Instagram: https://www.instagram.com/odsc/
• Blog: https://opendatascience.com/
• LinkedIn: https://www.linkedin.com/company/open-data-science/
• X (twitter): https://x.com/_odsc