Glossary · AI

What is
Prompt Caching?

API-level caching of prompt prefixes to reduce cost and latency on repeated calls.

← Glossary
By Anish· Founder · Vedwix
·

Definition

Prompt caching stores the prefix of a prompt server-side. When the same prefix is reused, you pay a fraction of the input-token cost and get faster responses. Anthropic and OpenAI both offer cached prompt mechanisms. Best practice: structure prompts with the static system instructions and large context first, dynamic user input last.

Example

A documentation chatbot caches its 5,000-token system prompt; subsequent calls cost ~10% of an uncached call.

How Vedwix uses Prompt Caching in client work

Always-on for any app with a substantial system prompt. Often a 30–70% cost saving.

Building with Prompt Caching?

We ship this.

If you're building with Prompt Caching in production, we can help — from architecture review to full implementation.

Brief us
Related terms
Prompt EngineeringThe practice of crafting LLM inputs to produce better, more reliable outputs.Context WindowThe maximum number of tokens an LLM can process in a single call.Total Cost of Ownership (TCO)The full cost of running an AI feature in production — inference, eval, observability, ops, and people.
More AI terms
RAGAIFine-tuningAIEmbeddingAIVector DatabaseAIHybrid SearchAIRerankerAI

Working on a Prompt Caching project?

Brief Vedwix in three sentences or fewer.

Start a project