📖 The Token Saving Series (4 parts): Part 1: Why your AI gets dumber the more you chat · Part 2: If an abacus works, don’t boot up a supercomputer · Part 3 (This post) · Part 4: I built a budget dashboard for my AI

The first two posts were still at the “everyone can use this” level. Starting with this one, I’ll show you the setup on my computer that I’ve endlessly tinkered with.

First, some background. Aside from chatting with AI on my phone, what I use more often is a tool called Claude Code: it runs in my computer’s terminal, can read my files directly, edit my notes, and automate a bunch of things for me. You can think of it as “an AI assistant living in my computer that can actually do things.”

Because it can do so much and runs so often (a single task can easily call the AI dozens or hundreds of times), the payoff for saving tokens here is magnified many times over. And the first principle of saving tokens is exactly the same as organizing a desk:

Only keep what you need right now on your desk, and put everything else in drawers.

The residency tax: a cost you pay with every single sentence

I have a file called CLAUDE.md. It contains my long-term rules for the AI: what format to use for my notes, which folders not to touch, and what my preferences are.

This file has a specific trait: it gets loaded right at the start of every session, and stays right in front of the AI for the entire workflow, never put away.

This means if this file has 5,000 tokens, then whether I only ask two sentences today or chat for two hundred, I’m repeatedly paying for those 5,000 tokens behind every single sentence.

I call it the “residency tax.” Every line is a tax.

So I deliberately compress it to about 95 lines, keeping only the truly stable, absolutely necessary rules. Meeting minutes, design processes, and long explanations are all moved elsewhere, only called up when needed.

By the way, I always write these “AI-only” config files in English. That’s because Chinese is more expensive in token calculation (Chinese expressing the same meaning usually eats up more tokens than English), while I use Chinese for notes meant for humans. Cost savings for the AI, readability for humans. Everyone wins.

Drawers: layer your memory, don’t spread it all on the desk

I built a memory system for my AI, but the point isn’t “the more it remembers, the better.” The point is layering:

  • What’s truly used every day goes on the top layer, loaded automatically at the start of every session.
  • What’s only used occasionally goes into drawers, pulled out only when needed.
  • What hasn’t been touched in a long time gets archived.

I also made a “usage heat” score: frequently read memories get a high score and stay on the desk; the cold ones automatically sink. This way, the desk always has only the most relevant papers for the current moment.

The logic is the same as the abacus in the last post: this heat sorting is calculated by code, without bothering the AI.

This is also a useful insight for people using the mobile chat version: ChatGPT’s “Memory” and “Custom instructions” features aren’t free magic. They’re invisible content automatically pasted in before every conversation, taking up space and being recalculated with every sentence just the same. So memory needs to be lean; cramming more in isn’t better.

An almost free ultimate move: make the AI “remember” what it read last time

This trick is called prompt caching. It’s a built-in mechanism from Claude, and the one I think is most worth letting more people know about.

The principle is this: if the content you send this time starts with a large chunk exactly the same as last time (like that resident config file, or that long document), the system doesn’t have to recalculate it. It just reads it directly from the cache. And the price of reading from the cache is only one-tenth of the normal price (saving about 90%).

But there’s an easy landmine to step on: it compares from the very beginning. If even a single word at the start changes, the entire cache after it is invalidated and billed fresh.

The most common trap is someone stuffing a “current time” or something else that changes every time at the top of that resident config file. As a result, because that time line is different every single time, the entire cache is trashed, and they pay ten times the price for nothing without even knowing it.

So the top of my config file is always stable, unchanging content. Anything that might change gets pushed to the back. This one small detail saves a considerable amount of money.

Two small contraptions I hooked up myself

Since Claude Code allows me to write “interceptors” to automatically handle things before and after the AI acts, I hooked up two that I really like:

Contraption 1: no nagging reports for successful commands. Every time the AI finishes running a command (like saving or moving a file), that command often spits out a huge string of “I did this and that” messages. If it succeeded, I don’t need to see any of this nonsense. So I set up an interceptor: when a command succeeds, throw its output string straight in the trash; only leave the message if there’s an error. The desk is instantly clean, and tokens are saved.

Contraption 2: no throwing PDF images directly to the AI. Echoing the last post, I wrote an interceptor that directly blocks the action of “sending a PDF page as an image to the AI,” forcing it to go through local text extraction first. Block the expensive path, and it has no choice but to take the cheap one.

Wrapping up: clean is cheap

All the tricks in this post really boil down to one sentence: only keep what the AI needs right now on its desk.

  • Keep config files lean, because they are a residency tax you pay on every sentence.
  • Layer memory: common stuff on the desk, cold stuff in drawers.
  • Use prompt caching well, so repeated content only costs a tenth of the price.
  • Throw away successful noise directly; don’t let it take up space.

A clean desk doesn’t just save money; the AI answers more accurately, too (remember “gets dumber the more you cram” from the first post? Same logic).

The next post is the final one in this series, and also the one I think is the most fun: how I pick the right level of brain for the AI, and how I built myself a budget dashboard that pops up to warn me “you’re burning money too fast today.”

—

⬅️ Previous post: If an abacus works, don’t boot up a supercomputer ➡️ Next post: I built a budget dashboard for my AI

References