There is a particular kind of incident every SRE team is familiar with. A common component of your stack, say your Redis database, starts misbehaving. Someone spends two hours tracing it back to a connection pool exhausted by a misconfigured client, the fix goes in, and everyone moves on, for today. The following Tuesday it happens again, and whoever is on call investigates it from scratch, because the person who solved it last week is asleep, on vacation, or working somewhere else now. The knowledge existed. It just lived in one engineer's head, and that kind of tribal knowledge never scales. Entire books like "The Phoenix Project" were written about this. Ask a good on-call engineer why the checkout service is flapping and they rarely start from nothing, they remember this tribal knowledge. They recall when this happened in March, that the culprit was a readiness probe, and the fix was a longer grace period. That memory is what separates a senior responder from a smart newcomer, and until recently, it was exactly what every AI investigator lacked. An AI SRE without memory has the same problem, only worse. However capable the model is, it is stateless. Run it on the same recurring failure a hundred times and it re-derives the same conclusion a hundred times, paying full price each round and never getting wiser about your specific environment. This is the problem Klaudia Memory was built to solve. In our recent post we introduced it as one of the context layers that power Klaudia, Komodor's autonomous AI SRE. In this post, we will take a closer look at how it works under the hood and the engineering that went into building the backbone of a true AI SRE platform. What Makes a Memory Good For memory to be worth anything, two critical pieces have to consistently work. It has to be saved and indexed correctly It has to be retrievable at exactly the moment it is needed. Miss the mark with either of these, and you have a write-only archive. Consider what actually happens during an investigation. A large share of any AI SRE session is exploration, reading logs, walking YAMLs, tracing routing, checking events, the same kubectl archaeology a human engineer would do. While it’s true that most of that exploration is only relevant once, buried inside are uniquely useful nuggets and findings that will shorten the exploration path the next time. It might be a dependency nobody documented, or that configuration that turned out to be the trigger, or ultimately the fix that actually held. After every investigation, Klaudia extracts exactly those pieces of information and indexes them for future sessions. Facts, Not Transcripts A naive approach to memory would store past investigations verbatim. That scales badly and leads to poor recall, because raw transcripts are long and full of redundant detail. Instead, each investigation is distilled into a small number of discrete facts, where a fact is a single unit of operational truth. Some examples: Service A depends on Service B. This deployment fails when that ConfigMap value drifts. Scaling this workload past a certain point exhausts the node pool. The extraction step is intentionally strict. It rejects generic truths, tautologies, and prescriptive advice. A fact earns its place only if it carries a concrete, environment-specific detail: a name, a threshold, a dependency, a measured duration. What survives is dense and durable, the kind of thing a teammate would actually tell you before you paged them at 2:17 AM. Every fact is classified into one of eight categories of operational knowledge, and together they map surprisingly well to how experienced SREs think about their systems. Structure Topology for resource dependencies and environment shape. Blast Radius for what breaks when something fails and how impact propagates. Capacity Patterns for scaling behavior and resource consumption. Temporal Patterns for recurrences such as deploy-day incidents, cron windows, and traffic peaks. Failure Correlations link the symptoms and components that co-occur. Resolution Playbooks record what fixed an issue, what didn't, and how. Tribal Knowledge covers ownership, escalation paths, and informal conventions. Operational Constraints records known hard limits, config baselines, and the things that break under very specific conditions. Indexing: One Record, Three Ways to Find It Storing a fact is easy. Storing it so it resurfaces at the right moment, in the middle of a live investigation, is the actual engineering problem. Each memory is therefore indexed with sufficient context to be found through more than one lens. Every memory carries the same set of structured metadata, the context that makes effective retrieval possible, from the resource it relates to through to the kind of operational knowledge it captures. The schema is constant across all memories, which is what gives each fact its perennial relevance and makes uniform filtering possible, while the values anchor every record to the specific environment it came from. On top of that sit two search indexes. The resource identity is indexed as full text for fast keyword matching on the entities involved The content of the memory itself is embedded as a dense vector that captures what it means, so conceptually related memories surface even when the words differ. That vector comes from a dedicated embedding model rather than an LLM, a deliberately separate choice, since mapping text to searchable vectors is a much narrower job than reasoning about it. Retrieval: Filter First, Search Second Pulling every memory into every investigation would be a waste, and worse, it would be noise. So retrieval is hybrid and works in layers. The first layer is metadata filtering. Before any search runs, the candidate set is narrowed by the context of the investigation itself, this cluster, this resource, this domain. Memories about an unrelated environment never enter the conversation. The second layer runs the searches in parallel, exact matching on the resource identity, keyword search over the indexed identity and metadata, and semantic search over the content embeddings. No single method is enough on its own. Exact match is precise but blind to paraphrase, while semantic search is fuzzy but can miss the literal resource, keyword search sits right in between. Each returns its own ranked candidates, the scores are normalized and fused, and the strongest memories rise. Raw relevance leads, but it is tempered by recency, because knowledge decays, and by reinforcement, because a fact confirmed across many incidents is worth more than a one-off observation. The retrieved memories are then synthesized into a grounded, past-tense answer, strictly conveying only what was observed and done before, never offering speculative advice. And the system records which memories actually shaped the answer, because that signal closes the learning loop. Facts that keep proving useful are reinforced. Facts that stop being reinforced fade rather than fossilize. That last part is the piece most teams building memory systems get wrong, because the honest risk with any learning layer is staleness. A failure correlation that was true twelve months ago may describe an architecture that no longer exists, and an outdated pattern that misleads an investigation is worse than no memory at all. So confidence decays, and stale patterns quietly sink in the rankings instead of resurfacing. Memories have to keep earning their place. The Cost of Good Memory Memory is queried constantly, at multiple points inside every investigation, and paying frontier-model prices for every one of those calls adds up fast. It is also unnecessary by design. Because the metadata filter and the fused search have already done the heavy lifting of narrowing candidates, the model consuming the results does not need frontier-scale reasoning. It needs to read a short list of highly relevant, pre-filtered facts and apply them. For that narrower job, smaller and far less costly models like Qwen, Kimi, DeepSeek, or Mistral are more than capable, and the team tested the pipeline against a range of them through careful prompt and context engineering to find the ones that hold up. The work that genuinely demands deeper intelligence, the investigative reasoning itself, still runs on stronger models like Claude Sonnet. Spending on intelligence where it matters and economy where it doesn't is what offsets the cost of an always-on memory, keeping the layer fast enough and cheap enough to be called liberally. That matters, because a memory that is expensive to consult is a memory that ends up unused. From Episodes to Tribal Knowledge that Compounds Individual facts are episodic, they record what happened, once. The real payoff comes when enough episodes accumulate and Klaudia distills them into semantic knowledge, durable truths about how your environment is actually shaped. That distillation is automatic and passive by design. Nobody writes it, curates it, or has to remember to save anything, which is exactly what separates Klaudia Memory from the other context layers. The Knowledge Base indexes the runbooks and postmortems your team already wrote. Klaudia.md captures the hard constraints your team defines once. Memory deposits its lessons on its own, investigation after investigation, oral tradition made persistent. It is also the one capability that appreciates over time. Most tooling is at its best on day one and ages from there, and human memory is only as good as the person who owns it. Klaudia inverts that curve, because learning quality is proportional to investigation volume, and investigation volume is a function of time. The longer Klaudia runs in your environment, the more of its specific operational truth it holds, from the dependencies nobody documented to the failure modes unique to your architecture. The most valuable thing a senior SRE brings to an incident has always been the accumulated pattern library of every outage they ever sat through. That library used to walk out the door with them. Now it persists, compounds, and is available to whoever is on call, at any hour, from the first minute of the investigation. —-- See Klaudia Memory in action. Book a demo or start a free trial today.
