Google unveils server-side AI memory design

Google on September 23 unveiled a design that adds server-side memory to Private AI Compute, its cloud AI processing platform. Until now the platform was stateless, wiping all context the moment a task ended; the plan is to keep each user's memories encrypted in the cloud so they carry across devices. Google said the data will be inaccessible to anyone else, including Google. The announcement appeared on the Google DeepMind blog, accompanied by a technical brief detailing the design, a public record of the server software and the results of an external audit.

Google first introduced Private AI Compute on November 11, 2025. According to reports, it runs on Google's own TPUs and lets Gemini models process sensitive data in a hardware-isolated environment. At the time, Google said the platform would make Magic Cue on Pixel 10 more helpful and widen the languages the Recorder app can summarize. In this announcement, the Google Private AI Compute team explained that "a new persistent memory layer will be able to function like a secure digital vault in the cloud."

The technical brief explains why memory is needed. Ephemeral inference cannot carry a user's long-running project, the decisions already made on their behalf or the preferences inferred in earlier sessions. On-device storage runs into its own walls: physical capacity limits, friction syncing across devices, and the bandwidth and latency cost of repeatedly sending large historical contexts to cloud models. Google concluded that workarounds, such as having the AI save lists of personal facts and preferences, cannot deliver the continuous experience people expect.

At the center of the design is the memory Oak Server, a per-user database running inside a hardware trusted execution environment. Records are encrypted under keys bound to the user, and those keys are never visible outside the trusted computing base or to Google's infrastructure. The surrounding platform can store, replicate and back up the encrypted database without ever reaching the plaintext. An orchestrator mediates between the AI model and the memory server, and user data never leaves the protected zone, the enclave, in plaintext. The memory application is written in Rust and runs on the Oak Containers runtime, and both the server and the runtime are open source.

The brief also lays out how a single request is handled. The device opens an encrypted session using the Noise Protocol, and the request goes to an orchestration enclave inside an AMD SEV-SNP confidential virtual machine. The orchestrator sets up a mutually attested ALTS channel with the memory server, and only after the hardware verifies the enclave's measurement are the user's decryption keys released. The relevant records are decrypted only in volatile enclave memory and merged with the prompt, and the model computes inside the TPU boundary. New memories or updated preferences are re-encrypted under the user's key and stored, and prompt context, tokens and intermediate activations are wiped as soon as the response is delivered.

The 15-page technical brief, which METAL reviewed, does not hide that retaining memory changes the nature of the risk. Google wrote that "persistent state carries a fundamentally higher risk profile than ephemeral computation," which is why it released the memory Oak Server as open source for public review. The stateless inference path is designed so a query cannot be linked to a specific user, but memory requires a request to reach that user's store, so a stable per-user identifier is needed. The brief therefore states that this system does not claim network-level non-targetability, and explains instead that targeting a specific user's memory yields only unreadable ciphertext.

The security goals for the persistent store are also specific. There is no administrative path to plaintext user data even in emergencies, and the memory server runs in a confidential virtual machine so that a compromise of one instance does not expose other users' state. Default-deny policies keep memory contents from leaking through monitoring, logs or core dumps, and an enclave that cannot present attestation matching an endorsed memory binary, whether a modified or unauthorized build, does not receive the keys. The published source is tied to the deployed binaries through reproducible builds, and the resulting digests are published in an append-only ledger.

Google's position is that verification stays open to outsiders. According to the brief, external auditors validated the design both at the initial release and for this memory update, and Google has published summaries of its 2025 and 2026 audit reports. The next steps it lists are client-side verification in which user devices check server attestation directly, a transparency log co-signed by independent third parties, broader reproducible build coverage and recurring external audits. The brief describes this as "moving from Google-asserted compliance toward client-side cryptographic verification."

Through an AI engineer's lens, the heart of this announcement is less the memory feature than where the keys live. The blog says the keys are held on the user's devices, while the brief says the keys are bound to the user and exist in plaintext only inside an attested enclave. METAL has reported on Perplexity's agent memory system, Brain, and competition in personal AI is shifting from what an assistant remembers to who can open that memory. Reopening on a laptop assembly instructions first seen through smart glasses is still an example Google offered, not a shipped feature. Until user devices can verify servers directly, trust in this design depends on how closely people examine the published source and audit reports.