Nick is a Senior Principal Software Engineer at Red Hat focused on scalable serving of large language models, and a committer for the vLLM open source project, the de-facto standard open source LLM serving engine for production workloads. He previously led the architecture and development of distributed machine learning infrastructure at IBM Research, supporting key IBM AI cloud products and services. He designed and implemented the Model-Mesh serving framework that supports hundreds of thousands of models, now a core component of the KServe open source project.
Session¶
vLLM: A success story of UC and industry collaboration in Open Source AI