Blog

How Exabase reduces your token spend

Exabase replaces raw context injection with precise retrieval. Fewer tokens, better answers, costs that scale flat instead of linearly.

The core principle is simple: you should only send tokens that help the model answer the current question. Everything else is waste, in cost and in quality, because models degrade with irrelevant context.

Most agent architectures do the opposite. They inject full conversation histories and top-K RAG chunks into every prompt, paying for thousands of tokens that the model will never use. Exabase fixes this at the infrastructure level through three mechanisms.

Memory extraction replaces conversation history

When a conversation is processed through the Memory API, Exabase extracts the facts that matter and stores them as structured memories. An 8,000-token conversation might produce 50 tokens of structured facts: user preferences, decisions, stated constraints.

On the next request, instead of injecting the full conversation history, you retrieve only the memories relevant to the current query. A typical retrieval returns 400 tokens of relevant context rather than 12,000 tokens of raw history.

The memory engine also resolves contradictions at write time. If a user was on the free plan in message 5 and upgraded in message 22, only the current state is stored. The model receives one clean fact instead of two conflicting chunks from different points in the conversation.

Memory retrieval stays nearly flat as conversations grow because you are retrieving relevant facts rather than proportional history. A 50-message conversation does not cost meaningfully more to query than a 10-message one.

Deep Search replaces chunk dumps

Standard RAG retrieves top-K chunks by vector similarity and injects all of them. Deep Search searches at the sub-document level and returns precise passages with location references.

The difference: 10 precise passages at roughly 180 tokens each gives you 1,800 tokens of highly relevant context. Standard top-5 RAG at 500 tokens per chunk gives you 2,500 tokens of approximately relevant context. The Deep Search result is smaller, more precise, and gives the model less noise to work through.

The precision parameter provides a direct lever on this tradeoff. Set it high for specific factual questions where one or two passages contain the answer. Set it low for broader research queries where coverage matters more. Either way, you are controlling your token spend per request rather than accepting whatever the vector store returns.

Results also come back with page numbers and timestamps, so your agent can cite sources without injecting entire documents into the context for the model to search through.

Memory lookups replace LLM calls

Some questions do not need a language model at all. What plan is this user on, what is their preferred language, what timezone are they in. These are lookups.

With structured memory, these become API calls that return in 200ms and consume zero LLM tokens:

javascript

const result = await exabase.memories.search({
  query: "user plan tier",
});
// → { content: "User is on Enterprise plan", score: 0.89 }

const result = await exabase.memories.search({
  query: "user plan tier",
});
// → { content: "User is on Enterprise plan", score: 0.89 }

const result = await exabase.memories.search({
  query: "user plan tier",
});
// → { content: "User is on Enterprise plan", score: 0.89 }

Each lookup that replaces an LLM inference eliminates an entire call's worth of token spend and latency. In a typical session there are several of these opportunities.

The combined effect

Here is what a single request looks like before and after:

Before:

System prompt:                        500 tokens
Full conversation history (20 msgs):  12,000 tokens
RAG chunks (top 5):                   2,500 tokens
User message:                         100 tokens
───────────────────────────────────────────────
Total:                                15,100 tokens

System prompt:                        500 tokens
Full conversation history (20 msgs):  12,000 tokens
RAG chunks (top 5):                   2,500 tokens
User message:                         100 tokens
───────────────────────────────────────────────
Total:                                15,100 tokens

System prompt:                        500 tokens
Full conversation history (20 msgs):  12,000 tokens
RAG chunks (top 5):                   2,500 tokens
User message:                         100 tokens
───────────────────────────────────────────────
Total:                                15,100 tokens

After with Exabase:

System prompt:                        500 tokens
Retrieved memories (relevant only):   400 tokens
Deep Search chunks (10 chunks):     1,800 tokens
User message:                         100 tokens
───────────────────────────────────────────────
Total:                                2,800 tokens

System prompt:                        500 tokens
Retrieved memories (relevant only):   400 tokens
Deep Search chunks (10 chunks):     1,800 tokens
User message:                         100 tokens
───────────────────────────────────────────────
Total:                                2,800 tokens

System prompt:                        500 tokens
Retrieved memories (relevant only):   400 tokens
Deep Search chunks (10 chunks):     1,800 tokens
User message:                         100 tokens
───────────────────────────────────────────────
Total:                                2,800 tokens

That is roughly an 81% reduction in input tokens per request.

At GPT-4o's input pricing ($2.50/M tokens), 10,000 requests per day at 15,100 tokens each costs approximately $1,133/month on input alone. At 2,800 tokens each, it is approximately $210/month. To estimate the savings for your specific workload, try the token cost calculator.

These numbers are illustrative and depend on conversation length, document corpus size, and query patterns. But the structural point holds: extracted memory and precise retrieval scale flat, while conversation stuffing and naive RAG scale linearly with usage.

Get started

javascript

import { Exabase } from "@exabase/sdk";

const exabase = new Exabase({ apiKey: process.env.EXABASE_API_KEY });

// Extract memories from a conversation
await exabase.memories.create({
  source: 'text',
  content: yourConversationHistory,
});

// Retrieve relevant context for the next request
const memories = await exabase.memories.search({
  query: incomingUserMessage,
  rerankThreshold: 0.5,
});

import { Exabase } from "@exabase/sdk";

const exabase = new Exabase({ apiKey: process.env.EXABASE_API_KEY });

// Extract memories from a conversation
await exabase.memories.create({
  source: 'text',
  content: yourConversationHistory,
});

// Retrieve relevant context for the next request
const memories = await exabase.memories.search({
  query: incomingUserMessage,
  rerankThreshold: 0.5,
});

import { Exabase } from "@exabase/sdk";

const exabase = new Exabase({ apiKey: process.env.EXABASE_API_KEY });

// Extract memories from a conversation
await exabase.memories.create({
  source: 'text',
  content: yourConversationHistory,
});

// Retrieve relevant context for the next request
const memories = await exabase.memories.search({
  query: incomingUserMessage,
  rerankThreshold: 0.5,
});

The clearest way to validate this is your own token logs before and after. For the full technical walkthrough of the three patterns, see how to cut token costs. For the underlying problem these patterns solve, see how you're overspending on tokens.

Exabase docs · Memory API · Deep Search · Token cost calculator