Repository navigation
About Concurrency and Stability #601
Description
Feature request
Hello! I recently came across this popular OpenAI-compatible inference framework and found it very interesting. I'd like to know more about its concurrency and stability—specifically, how it compares to vLLM.
Motivation
project feature
Your contribution
no
Activity
michaelfeil commentedon Aug 22, 2025
https://www.baseten.co/blog/how-we-built-bei-high-throughput-embedding-inference/#performance-benefits-from-baseten-infrastructure Answer: It depends. It has much more features arond embeddings than vllm.