glossarytech.com

vLLM

A system designed to run large language models efficiently by managing memory and requests smartly. It is often used to serve modern machine learning models at scale.

First released2023
Developed byUC Berkeley
Open-sourceYes

Interesting facts

vLLM is known for improving throughput and reducing costs when serving large language models.

Sign up for updates
straight to your inbox

Subscribe