When AI Knows Everything About You: The Problem of an Empty Promise

In early June, journalists David Pierce and Jay Peters from The Verge had an exclusive trial of Spark, Google's new AI agent based on Gemini. Their findings converged on one uncomfortable point: the agent knew Pierce's dog's name—Frida—and the first name of Peters' wife, even though neither had explicitly provided this information during the session. There was no hallucination, no error. Spark simply knew.

That seemingly minor detail opens a question more serious than any performance benchmark: at what cost do we buy utility?

Effectiveness as a Symptom

For years, the argument for adopting AI assistants was that they were tools, not actors. You gave them context, they processed. The boundary between what the system knew and what the user had given it was, in theory, clear.

What Spark shows—and what we're increasingly seeing in agents built on Claude Opus 4.7 with 1 million token context windows—is that this boundary has become porous. Modern agents don't just process what you tell them in a session: they access emails, calendars, browsing history, contacts, and previous conversations to build an operational profile that allows them to act with an effectiveness that, as Pierce and Peters aptly describe, feels unsettling.

The problem isn't technical. It's contractual and cultural.

The Promise Nobody Read

When a user activates an AI agent with broad permissions, they accept—often without reading it—that the system will access their data ecosystem. The privacy policies of these products have warned about this in fine print for years. What's new isn't that it happens: what's new is that it's now visible. The agent no longer fails to retrieve that personal data; it uses it fluently, the way a human assistant who'd been by your side for years would.

That visibility is what reveals the empty promise mentioned in The Verge's headline. The implicit promise was: AI helps you without knowing too much about you. What we have in 2026 is the opposite: AI helps you precisely because it knows a lot about you.

Who This Matters to and How Much

For home users who use these agents in personal mode, the discomfort is mainly psychological. The question is whether they're willing to accept that intimacy in exchange for utility.

For teams deploying agents in business environments—something we see regularly at ElephantPink in integrations with Claude Code and MCP servers connected to internal data sources—the problem is more structural. An agent with access to corporate email, repositories, and CRM can leak sensitive information in ways traditional access controls don't anticipate. Not through malice, but by design: that's exactly its differentiating value.

Hooks and `PreToolUse` policies in Claude Code exist, in part, to mitigate this: they let you audit and block tool calls before they execute. But they're opt-in controls requiring active configuration, not default behaviors.

The Debate Ahead

The Verge article doesn't level an accusation; it identifies a tension. As agents improve, the friction that once served as an informal safeguard—the agent couldn't find the data, or failed to use it—disappears. What remains is the real decision: how much personal context is acceptable to surrender for a truly useful agent.

Regulators and informed users aren't making that decision yet. Product teams at Google, Anthropic, Microsoft, and others are making it, through the permissions they request by default and those they don't.

In our view, the moment to establish clear standards for granular consent on agents with persistent access to personal data isn't when something goes wrong. It's now, while the debate still fits in an opinion piece rather than a legal filing.