The hidden cost of inference: what AI leaves stored on disk

An eight-second AI-generated video weighs 3.97 MB when it reaches the user. The systems that produced it are left holding more than 500 MB of data: intermediate versions, metadata, logs and discarded outputs. That is a ratio of roughly 130 to one, and it is the figure that opens the article published on 22 October in MIT Technology Review about the storage bill that inference leaves behind.

It is worth reading with its origin in mind. This is sponsored content, produced by Western Digital (WD) and not by the magazine's newsroom, as the page itself states. A drive maker arguing that more storage is needed surprises nobody. Even so, the data it presents points to a real cost that many teams are not budgeting for.

What the article says

The argument is simple. When a company estimates what AI costs, it thinks of GPUs, power consumption and waiting lists for chips. What is missing from that picture is what happens to the data after the model responds. Every prompt, every image and every video leaves a trail in the back end, and much of that trail is kept: for auditing, for debugging, for retraining or to feed retrieval systems.

The figures in the piece:

According to McKinsey projections, inference demand will grow at a 35% compound annual rate through 2030, compared with 22% for training.
By 2030, inference would account for more than 40% of total data center demand.
An IDC white paper, commissioned by WD and based on a survey of 763 IT and business decision makers, finds that 95% have seen their data volume grow because of generative AI in the past 12 months.
61% report growth above 25%.
96% expect to need faster retrieval of archived data for inference and RAG.

The piece also quotes a product manager at a large cloud provider, without naming them: "You can think of having more storage as an optimization for keeping the GPUs busy."

Why it matters

The shift in spending from training to inference is no longer a hypothesis. Training a model is a one-off cost, huge but bounded. Serving it is a cost that repeats with every request. If every request leaves data that someone decides to keep, storage starts growing with product usage rather than with the number of models.

For anyone integrating Claude or other LLMs in production, this translates into very concrete questions. How long conversations are kept. Whether an agent's intermediate outputs are stored. Where the embeddings of a RAG system live and how often they are queried. An agent that chains tool calls through MCP servers generates far more of a trail than a question-and-answer chatbot, because every call, its result and the intermediate steps can end up in the logs.

Who it is useful for

The piece is most valuable for these profiles:

Platform and FinOps teams working out the cost of moving an AI pilot into production.
Product owners with image or video generation features, where the gap between what the user sees and what gets stored is widest.
Anyone designing RAG systems, because fast access to archived data becomes part of the latency the user perceives.

What the article does not offer is a method for calculating that cost or a comparison between storage tiers (hot, cold or archive). Nor does it separate data that must be kept for legal reasons from data kept out of inertia, and that second category is usually the easiest to cut.

What we see when integrating agents

In integrations with agents and MCP servers, storage rarely shows up in the first budget. It tends to appear a few months later, when traces, transcripts and caches start taking up more space than the code itself. The answer almost never involves buying more disk, but deciding from the start what gets kept, for how long and on which tier.

The fact that the warning comes from a drive maker does not invalidate it, but it does mean reading it with a pencil in hand. A data retention policy should enter the design of an AI product on the same day the model is chosen, not when the first invoice arrives.