50 Ways to Reduce LLM Token Usage

Master token reduction beyond simple prompting—50 practical techniques across prompt engineering, RAG, code context, and data compression.

By Sea Wing AI
Enterprise AI LLMtoken usagecontext engineeringprompt engineeringcost optimization

Where the biggest gains for low cost LLM usage come from?

The biggest gains rarely come from making prompts slightly shorter. They come from context engineering: retrieving only the smallest high-signal context needed for the task, avoiding repeated transmission of the same information, and using structured representations rather than raw text.

Everyone talks about better prompts, but the biggest savings in LLM costs often come from better context engineering.

Graphify

Recently, I shared Graphify, which transforms your codebase into a deterministic knowledge graph. Instead of sending thousands of lines of code to the LLM every time, the AI navigates the graph and loads only what’s relevant.

That’s a great example of structural context optimization—and it’s only one piece of the puzzle.

Here are 50 additional ways to reduce LLM token usage, organized by category:

1. Prompt Engineering

  1. Remove unnecessary words and explanations.
  2. Replace paragraphs with bullet points.
  3. Use consistent abbreviations.
  4. Remove repeated instructions.
  5. Move permanent instructions into the system prompt.
  6. Reference previous answers instead of repeating them.
  7. Use IDs instead of long names.
  8. Request structured JSON instead of verbose prose.
  9. Limit output length explicitly.
  10. Avoid examples unless necessary.

2. Conversation Management

  1. Summarize conversations every few turns.
  2. Keep only the relevant chat history.
  3. Compress old messages into memory.
  4. Store conversation state externally.
  5. Replace long history with structured facts.
  6. Remove greetings and pleasantries.
  7. Delete failed attempts.
  8. Reset context after task completion.
  9. Split unrelated topics into separate chats.
  10. Store reusable facts in a profile.

3. RAG Optimization

  1. Retrieve only the top-ranked chunks.
  2. Rerank before sending context.
  3. Merge overlapping chunks.
  4. Remove duplicate passages.
  5. Extract only relevant paragraphs.
  6. Compress retrieved documents.
  7. Filter by metadata first.
  8. Retrieve summaries before full documents.
  9. Use hierarchical retrieval.
  10. Retrieve only changed documents.

4. Code Context Optimization

  1. Send ASTs instead of full source files.
  2. Load only relevant functions.
  3. Use symbol lookup.
  4. Send interface definitions only.
  5. Replace implementations with signatures.
  6. Send Git diffs instead of entire files.
  7. Load dependency graphs first.
  8. Use semantic indexes such as Graphify.
  9. Reference file paths instead of embedding file contents.
  10. Cache parsed project metadata.

5. Data Compression

  1. Use compact data formats.
  2. Remove unnecessary whitespace.
  3. Shorten property names.
  4. Deduplicate repeated values.
  5. Encode repeated structures once.
  6. Replace repeated text with references.
  7. Convert tables to CSV.
  8. Use domain-specific mini languages (DSLs).
  9. Send hashes or IDs for unchanged data.
  10. Store large assets externally and retrieve them only on demand.

Highest ROI

  1. Graphify / Knowledge Graphs
  2. AST-based retrieval
  3. Context summarization
  4. Prompt caching
  5. RAG re-ranking
  6. Conversation compression
  7. Delta prompting
  8. Symbol/function retrieval
  9. Semantic caching
  10. Structured output

For teams building AI applications, there are even more advanced techniques:

n Prompt cachingn Semantic caching n Delta promptingn Hybrid retrieval (BM25 + embeddings) n Model routingn Multi-stage pipelines n Knowledge graphsn Context deduplication n Entity extractionn Dynamic context windows

Where you’ll usually see the biggest savings

n Graph-based retrieval & knowledge graphsn AST-based code retrieval n Context summarizationn Prompt & semantic caching n Conversation compressionn Delta prompting `n Function/symbol retrieval

Highest ROI (rough order)

TechniqueTypical Savings
Graphify / Knowledge Graph10×–70×
AST-based retrieval5×–30×
Context summarization3×–20×
Prompt caching50–95% repeated prompts
RAG re-ranking30–80%
Conversation compression50–90%
Delta prompting70–99%
Symbol/function retrieval80–99%
Semantic cache70–100% cache hits
Structured output20–60%

One lesson I’ve learned while building AI systems:

The biggest reductions in cost don’t come from writing slightly shorter prompts. They come from sending dramatically less context.

The future of AI isn’t just prompt engineering—it’s context engineering.

What techniques have given you the biggest reduction in token usage?

← Back to Blog
Discuss Now