# vLLM Agent Docs

> An unofficial, community-run migration of a focused subset of vLLM's documentation onto the Thally docs platform

- Documentation: https://vllm-agent-docs.sudhanvasp.dev
- GitHub: https://github.com/sudhanvasp/vllm-agent-docs
- Full docs for LLMs: https://vllm-agent-docs.sudhanvasp.dev/llms-full.txt
- Agent discovery (ai.txt): https://vllm-agent-docs.sudhanvasp.dev/ai.txt
- Structured docs index (JSON): https://vllm-agent-docs.sudhanvasp.dev/api/docs-index
- OpenAPI spec: https://vllm-agent-docs.sudhanvasp.dev/openapi.yaml

## For AI agents

Every page supports content negotiation on its human URL:
- JSON API: append `?format=json` or send `Accept: application/json`
- JSON-LD: append `?format=ldjson` or send `Accept: application/ld+json`
- Markdown: append `?format=md` or send `Accept: text/markdown`
- Per-page API: https://vllm-agent-docs.sudhanvasp.dev/api/docs/{page-id}
- Search API: https://vllm-agent-docs.sudhanvasp.dev/api/search?q={query}
- Capability manifest (skill): https://vllm-agent-docs.sudhanvasp.dev/skill.md
- Agent guidance (editing these docs): https://vllm-agent-docs.sudhanvasp.dev/AGENTS.md
- MCP server (attach docs as native tools): https://vllm-agent-docs.sudhanvasp.dev/api/mcp
- Agent readiness: https://vllm-agent-docs.sudhanvasp.dev/api/agent-readiness

### Recommended workflow

1. Use this index, the search API, or `search_docs` before choosing a page.
2. Read the smallest set of relevant pages and follow their prerequisite links.
3. Treat the published docs as the source of truth. Label inferences and say when evidence is missing.
4. Cite the canonical human page URLs, not only API endpoints.
5. Use `/AGENTS.md` before editing a repository. The public MCP server is read-only.

## Get started

### Get started

- [Introduction](https://vllm-agent-docs.sudhanvasp.dev/): An unofficial, agent-ready mirror of vLLM's serving documentation, migrated from docs.vllm.ai.
- [Quickstart](https://vllm-agent-docs.sudhanvasp.dev/quickstart): Install vLLM and serve your first model with one command.
- [Troubleshooting](https://vllm-agent-docs.sudhanvasp.dev/troubleshooting): Diagnose the most common vLLM failures — out-of-memory errors, hangs, NCCL errors, and unsupported models.
- [FAQ](https://vllm-agent-docs.sudhanvasp.dev/faq): Answers to the questions vLLM users ask most.

### Guides

- [Serve an OpenAI endpoint](https://vllm-agent-docs.sudhanvasp.dev/guides/serve-openai-endpoint): Stand up vLLM's OpenAI-compatible server and call it with curl and the official OpenAI Python client.
- [Distributed serving](https://vllm-agent-docs.sudhanvasp.dev/guides/distributed-serving): Tensor parallelism vs. pipeline parallelism in vLLM, and when to combine them.
- [Docker & Kubernetes](https://vllm-agent-docs.sudhanvasp.dev/guides/docker-kubernetes): Run vLLM's official container image, then deploy it to a Kubernetes cluster.

## Reference

### Configuration

- [Engine arguments](https://vllm-agent-docs.sudhanvasp.dev/reference/engine-args): The engine arguments that control how vLLM loads, batches, and parallelizes a model.
- [Server arguments](https://vllm-agent-docs.sudhanvasp.dev/reference/server-args): How vllm serve CLI flags, YAML config files, and precedence work together.

### Models

- [Supported models](https://vllm-agent-docs.sudhanvasp.dev/reference/supported-models): Model families vLLM supports natively, and how to check whether a specific model works.
- [Quantization](https://vllm-agent-docs.sudhanvasp.dev/reference/quantization): The quantization methods vLLM supports, what each trades off, and how to enable one.

## API Reference

Interactive API reference available at https://vllm-agent-docs.sudhanvasp.dev/api