---
title: Introduction
description: An unofficial, agent-ready mirror of vLLM's serving documentation, migrated from docs.vllm.ai.
url: https://vllm-agent-docs.sudhanvasp.dev/
---

# Introduction

An unofficial, agent-ready mirror of vLLM's serving documentation, migrated from docs.vllm.ai.

> **Note:**
  This site is an independent, community-run migration of a subset of
  [vLLM](https://github.com/vllm-project/vllm)'s documentation onto the
  [Thally](https://thally.io) platform. It is **not** affiliated with, endorsed
  by, or maintained by the vLLM project or the PyTorch Foundation. For
  authoritative, complete, and current documentation, use
  [docs.vllm.ai](https://docs.vllm.ai).

## Start here

#### [Quickstart](/quickstart)

    Install vLLM and launch your first `vllm serve` command in a few minutes.

#### [Serve an OpenAI-compatible endpoint](/guides/serve-openai-endpoint)

    Stand up the OpenAI-compatible server and call it with curl or the OpenAI Python client.

#### [Troubleshooting](/troubleshooting)

    Diagnose out-of-memory errors, hangs, and unsupported-model failures.

#### [API reference](/api)

    Browse the OpenAI-compatible endpoints — chat completions, completions, embeddings, and models.

## Reference material

#### [Engine arguments](/reference/engine-args)

    The most commonly used `EngineArgs` flags: memory, parallelism, dtype, and quantization.

#### [Server arguments](/reference/server-args)

    How `vllm serve` CLI flags, YAML config files, and precedence work together.

#### [Supported models](/reference/supported-models)

    Model families vLLM supports natively, and how to check a specific model.

#### [Quantization](/reference/quantization)

    AWQ, GPTQ, FP8, GGUF, and BitsAndBytes — what each trades off and how to enable it.

## Scale and deploy

#### [Distributed serving](/guides/distributed-serving)

    Tensor parallelism vs. pipeline parallelism, and when to combine them.

#### [Docker & Kubernetes](/guides/docker-kubernetes)

    Run the official `vllm/vllm-openai` image, then deploy it to a cluster.

#### [FAQ](/faq)

    Answers to the questions vLLM users ask most.

#### [Changelog](/changelog)

    What changed on this site, generated from vLLM's GitHub releases.

## Why this site exists

This is a Track 2 hackathon submission demonstrating agent-ready documentation:
the same reviewed content is published as HTML, Markdown, JSON, JSON-LD, search,
an agent discovery index, and a read-only MCP server — so both people and AI
coding agents can find the same current answer. See the
[serving guide](/guides/serve-openai-endpoint) for the page verified across
every one of those surfaces.

## Hackathon reflection

> **Note:**
  Built with [Claude Sonnet 5](https://claude.com/claude-code) as a build
  partner throughout — content migration, verification, and this reflection
  were drafted with its help.

### What did you accomplish?

Migrated 12 scoped pages of vLLM's OpenAI-serving docs (Apache-2.0) onto
Thally, replacing all starter boilerplate: quickstart, a showcase "serve an
OpenAI-compatible endpoint" guide, troubleshooting, a generated changelog, a
scoped OpenAPI reference, engine/server args, supported models,
quantization, distributed serving, Docker/K8s, and FAQ. Pushed to GitHub and
connected to Thally Cloud on a custom domain, with an agent-readiness fix
applied and verified along the way.

### When did Thally first become useful?

Running `npm run check:agents` and getting back a numeric score with
per-signal subscores and a named offender (`/faq` — no headings) instead of
generic "improve your docs" advice. That's the moment it stopped feeling
like a docs template and started acting like a linter for agent-readiness —
fix the thing it named, rerun, confirm the number moved.

### What took more manual work than expected?

Custom-domain hosting could be smoother — connecting an existing GitHub repo
to Thally Cloud and pointing a Cloudflare-managed subdomain at it is a
web-dashboard, credential-gated flow with no CLI shortcut, so it took more
manual back-and-forth than the rest of the build.

### What result did you trust least?

The agent-readiness score jump itself (99 → 100). A single number
summarizing "is this agent-ready" is easy to satisfy on paper — add a
heading — without confirming an agent actually benefits from the change.

### How did you verify that result?

Didn't take the score's word for it: independently curled every surface the
showcase page claims to support (HTML, `.md`, the JSON API, embedded
JSON-LD, `/llms.txt` and `/api/docs-index`), then registered a real local
MCP server, called `tools/list` and `read_page` over JSON-RPC, and diffed
what came back against the Markdown mirror by hand. Also ran
`thally check --drift`, which independently caught a real broken anchor
link I hadn't been looking for — a better trust signal than the score alone.

### Would you use Thally for your next real release? Why or why not?

Yes, conditionally. The core value — one MDX source publishing consistently
to HTML, Markdown, JSON, JSON-LD, search, and MCP with no extra plumbing —
genuinely worked and removes a whole class of "the docs bot's answer
drifted from the docs page" bugs. The caveats are real, though: the Windows
CLI has a rough edge (a subprocess-spawn bug when Node lives under a path
with a space), and the readiness score is a fast lint, not a substitute for
someone — or some agent — actually trying to use the docs, which is what
caught the things the score missed.

## License

vLLM's documentation is licensed [Apache-2.0](https://github.com/vllm-project/vllm/blob/main/LICENSE)
by the vLLM project (a PyTorch Foundation project). Pages on this site are
adapted from that source under the same license; see each page for its origin.
This site's own scaffolding is provided by the [Thally](https://thally.io) starter.