Video by CNCF [Cloud Native Computing Foundation] via YouTube
The tokens are free. The GPU time, not so much.
At KubeCon + CloudNativeCon North America, William Morgan of Buoyant will explain how to build a practical, scalable and multi-tenant LLM token factory on Kubernetes using open source technology.
Learn about model selection, inference servers, authorization, rate limiting, quality of service, concurrency, and serving capable open weight models reliably at scale.
Join us November 9-12 in Salt Lake City.
Explore the Session: https://bit.ly/4dNo2u2
Register: https://bit.ly/3BDI5XL