[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"$fynnqmb4m91t0":3},{"code":4,"message":5,"data":6,"timestamp":72},10000,"success",{"full_name":7,"position":8,"description":9,"language":10,"stars":11,"forks":12,"stars_gained":13,"age_days":14,"hn_threads":15,"one_liner":42,"summary_md":43,"topic":44,"github_topics":47,"license":68,"homepage":69,"updated_at":70,"history":71},"vllm-project\u002Fvllm",6,"A high-throughput and memory-efficient inference and serving engine for LLMs","Python",95322,23894,1363,1353,[16,22,28,33,38],{"id":17,"title":18,"points":19,"comments":20,"posted_at":21},49492067,"vLLM v0.28.0",108,43,"2026-08-29",{"id":23,"title":24,"points":25,"comments":26,"posted_at":27},47643924,"vLLM introduces memory optimizations for long-context inference",5,0,"2026-04-04",{"id":29,"title":30,"points":31,"comments":26,"posted_at":32},47681076,"vLLM IR: A Functional Intermediate Representation for vLLM",4,"2026-04-07",{"id":34,"title":35,"points":36,"comments":26,"posted_at":37},49625693,"vLLM 0.29.0: Model Runner V2 is now the default",3,"2026-09-09",{"id":39,"title":40,"points":36,"comments":26,"posted_at":41},37453478,"Vllm: High-throughput and memory-efficient inference and serving engine for LLMs","2023-09-10","Serves and batches LLM inference at high throughput for teams running their own models.","## The problem\n\nOnce you move past a demo, the bottleneck stops being the model and becomes the serving layer: how many concurrent requests fit on a GPU, how you handle long prompts, and how you keep latency predictable while the KV cache grows. Naive PyTorch generation loops leave most of the hardware idle. vLLM is an inference and serving engine built to answer that question, with PagedAttention for KV memory management and continuous batching of incoming requests as its foundation.\n\n## What's different\n\nCompared with Hugging Face `transformers` generation, which is simple but not a serving system, vLLM is engineered for concurrency: paged KV cache, chunked prefill, prefix caching, CUDA\u002FHIP graphs, and a deep bench of quantized kernels. Compared with closed serving stacks, it is open and model-agnostic — the README claims 200+ Hugging Face architectures across dense, MoE, hybrid Mamba, multimodal, embedding and reward models, plus an OpenAI-compatible API and Anthropic Messages\u002FgRPC endpoints. The notable trade-offs are breadth of hardware (NVIDIA, AMD, Intel GPUs, CPUs, plus plugins for TPU, Gaudi, Ascend and others) and an unusually wide kernel surface; that breadth is the point, but it also means the internal behavior depends heavily on your exact GPU and model combo.\n\n## Who it's for\n\nTeams that host their own LLMs and need throughput per dollar: platform and MLOps engineers standing up an inference service, researchers benchmarking models, and anyone running MoE or multimodal models that most engines handle awkwardly. Look elsewhere if you only consume an API and never run weights, or if you need a supported appliance rather than a project you operate yourself.\n\n## Cost of adoption\n\nYou need Python, an accepted PyTorch\u002FCUDA (or ROCm) toolchain, and GPU memory sized to the model plus KV cache; installation is `uv pip install vllm`. Expect a sizeable dependency tree, version pinning against your driver, and a startup path that benefits from the precompiled kernels (`vllm download-kernels`). Apache-2.0 licensing is permissive, but the project is fast-moving: APIs and flags change between minor releases, and community activity around it in the HotRepo window hints at release churn rather than stagnation.\n\n## Verdict\n\nvLLM is not new to this category; it is one of the projects that defined it. Created in 2023-02, it has accumulated roughly 95k stars and 24k forks, and the 2026-08 Hacker News thread on v0.28.0 shows a project still shipping significant releases. Its +1,363 stars in a single week (weekly board #15) is healthy momentum, though it never appeared on the daily trending board during the tracked window — interest here is steady and engineering-driven, not a viral spike. This is a mature, high-commitment dependency: adopt it if you are serious about self-hosted inference, not as a weekend experiment.",{"slug":45,"name":46},"machine-learning","Machine Learning",[48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67],"amd","blackwell","cuda","deepseek","deepseek-v3","gpt","gpt-oss","inference","kimi","llama","llm","llm-serving","model-serving","moe","openai","pytorch","qwen","qwen3","tpu","transformer","Apache-2.0","https:\u002F\u002Fvllm.ai","2026-10-24T18:07:06Z",[],1792879059]