Agent Memory Patterns: How to Give Your LLM Application a Brain That Persists

I got a bug report six months into a production AI assistant deployment. A user had spent forty minutes in a conversation explaining their situation — the regulatory context, their organisation's specific constraints, the edge cases that mattered for their workflow. They closed the tab. The next day they came back, opened a new session, and the AI greeted them like a stranger.

They filed it as a bug. It wasn't a bug. It was the default behaviour of every LLM — a clean slate on every call. The model had no idea who they were, what they'd discussed, or what decisions they'd already made together.

We'd built a sophisticated RAG pipeline, a carefully tuned system prompt, a multi-step reasoning chain — and we'd completely ignored the most human thing about a useful assistant: it remembers you.

This is the problem every production AI application hits eventually. Here's how to actually solve it.

Why LLMs Forget Everything

An LLM has no persistent state. Every API call is stateless — the model receives a prompt, generates a response, and the weights return to their starting state. No trace of the interaction is retained anywhere inside the model.

What looks like "memory" in a multi-turn conversation is really you (the application developer) feeding the conversation history back into the context window on each call. The model isn't remembering — it's reading a transcript you reconstructed.

This works fine for short, contained conversations. It breaks in three specific production scenarios:

1. Long conversations — context windows are finite. A three-hour support conversation won't fit.
2. Cross-session continuity — a user returns the next day expecting to pick up where they left off.
3. Multi-user personalization — the system should know user preferences, history, and context without re-explaining from scratch.

Each scenario needs a different memory pattern. Getting the wrong one is common.

The Four Memory Types

Before picking an implementation, it helps to understand what "memory" actually means in an LLM system:

Most teams implement only in-context memory and wonder why the experience feels hollow.

Pattern 1: In-Context Memory (The Baseline)

The simplest form. Your application maintains a list of conversation turns and includes them in every request. Spring AI handles this out of the box:

@Service
public class ChatService {

    private final ChatClient chatClient;
    private final Map<String, List<Message>> sessionHistory = new ConcurrentHashMap<>();

    public ChatService(ChatClient.Builder builder) {
        this.chatClient = builder.build();
    }

    public String chat(String sessionId, String userMessage) {
        var history = sessionHistory.computeIfAbsent(sessionId, k -> new ArrayList<>());

        history.add(new UserMessage(userMessage));

        var response = chatClient.prompt()
            .system("You are a helpful assistant.")
            .messages(history)
            .call()
            .content();

        history.add(new AssistantMessage(response));

        // Critical: cap history to avoid context overflow
        if (history.size() > 40) {
            history = history.subList(history.size() - 40, history.size());
            sessionHistory.put(sessionId, history);
        }

        return response;
    }
}

When it works: Single-session conversations, stateless deployments where you don't need cross-session continuity.

When it fails: The moment the user closes the tab, the history disappears. In-memory storage doesn't survive restarts. And at scale, holding conversation history in a ConcurrentHashMap is a memory leak.

The if (history.size() > 40) check is the first mistake most teams skip. Without it, a long conversation will overflow the context window with a 400 error that looks like a bug, not a design problem.

Pattern 2: Episodic Memory with Redis

For cross-session continuity, you need the conversation history to survive restarts and be retrievable by session ID. Redis is the right tool — fast, TTL-aware, and mature.

@Configuration
public class RedisMemoryConfig {

    @Bean
    public RedisConnectionFactory redisConnectionFactory() {
        return new LettuceConnectionFactory("localhost", 6379);
    }

    @Bean
    public RedisTemplate<String, List<Message>> messageTemplate(RedisConnectionFactory factory) {
        var template = new RedisTemplate<String, List<Message>>();
        template.setConnectionFactory(factory);
        template.setKeySerializer(new StringRedisSerializer());
        template.setValueSerializer(new Jackson2JsonRedisSerializer<>(List.class));
        return template;
    }
}

@Service
public class PersistentChatService {

    private static final String KEY_PREFIX = "chat:session:";
    private static final Duration SESSION_TTL = Duration.ofHours(24);
    private static final int MAX_TURNS = 20;

    private final ChatClient chatClient;
    private final RedisTemplate<String, Object> redisTemplate;

    public String chat(String userId, String sessionId, String userMessage) {
        String key = KEY_PREFIX + userId + ":" + sessionId;

        // Load existing history from Redis
        List<Message> history = loadHistory(key);
        history.add(new UserMessage(userMessage));

        String response = chatClient.prompt()
            .system(buildSystemPrompt(userId))
            .messages(history)
            .call()
            .content();

        history.add(new AssistantMessage(response));

        // Trim and persist
        if (history.size() > MAX_TURNS * 2) {
            history = history.subList(history.size() - MAX_TURNS * 2, history.size());
        }

        saveHistory(key, history, SESSION_TTL);
        return response;
    }

    private List<Message> loadHistory(String key) {
        Object stored = redisTemplate.opsForValue().get(key);
        return stored != null ? (List<Message>) stored : new ArrayList<>();
    }

    private void saveHistory(String key, List<Message> history, Duration ttl) {
        redisTemplate.opsForValue().set(key, history, ttl);
    }
}

The userId + sessionId key structure matters. It gives you two retrieval options: fetch a specific session (precise, for resuming a conversation), or list all sessions for a user (broader, for showing conversation history in a UI).

TTL of 24 hours is a starting point. Healthcare conversations with compliance requirements might need 90 days. An anonymous support chat might need 2 hours. This is a business decision, not a technical default.

Pattern 3: Semantic Memory with pgvector

Episodic memory handles "what did we talk about in this session." Semantic memory handles "what does the system know about this user that should inform every interaction."

This is where things get genuinely powerful — and where the architecture gets more complex. The idea: after each conversation, extract meaningful facts about the user and store them as vectors. On future requests, retrieve the relevant facts and inject them into the context before the conversation history.

@Component
public class UserMemoryExtractor {

    private final ChatClient extractionClient;
    private final VectorStore vectorStore;

    private static final String EXTRACTION_PROMPT = """
        Analyse this conversation and extract durable facts about the user.
        Return ONLY a JSON array of facts. Each fact must be:
        - Specific and verifiable (not vague like "user is interested in AI")
        - Durable (still true tomorrow, not "user just asked about X")
        - Useful for personalising future responses
        
        Return [] if no durable facts are present.
        
        Example output:
        [
          "User's organisation is in the financial services sector",
          "User uses Spring Boot 3.2 with Java 21",
          "User has a compliance requirement to retain logs for 7 years"
        ]
        """;

    public void extractAndStore(String userId, List<Message> conversation) {
        String conversationText = conversation.stream()
            .map(m -> m.getMessageType() + ": " + m.getContent())
            .collect(Collectors.joining("\n"));

        String factsJson = extractionClient.prompt()
            .system(EXTRACTION_PROMPT)
            .user(conversationText)
            .call()
            .content();

        List<String> facts = parseFacts(factsJson);

        // Store each fact as a separate document with user metadata
        List<Document> documents = facts.stream()
            .map(fact -> Document.builder()
                .content(fact)
                .metadata(Map.of(
                    "userId", userId,
                    "extractedAt", Instant.now().toString(),
                    "type", "user_fact"
                ))
                .build())
            .toList();

        if (!documents.isEmpty()) {
            vectorStore.add(documents);
        }
    }
}

@Service
public class PersonalisedChatService {

    private final ChatClient chatClient;
    private final VectorStore vectorStore;
    private final UserMemoryExtractor memoryExtractor;

    public String chat(String userId, String userMessage) {
        // Retrieve relevant user facts for this query
        List<Document> userFacts = vectorStore.similaritySearch(
            SearchRequest.builder()
                .query(userMessage)
                .topK(5)
                .filterExpression("userId == '" + userId + "' && type == 'user_fact'")
                .build()
        );

        String systemPrompt = buildPersonalisedSystemPrompt(userFacts);

        String response = chatClient.prompt()
            .system(systemPrompt)
            .user(userMessage)
            .call()
            .content();

        // Async extraction — don't block the response
        // In production, push to a queue rather than calling directly
        CompletableFuture.runAsync(() ->
            memoryExtractor.extractAndStore(userId, List.of(
                new UserMessage(userMessage),
                new AssistantMessage(response)
            ))
        );

        return response;
    }

    private String buildPersonalisedSystemPrompt(List<Document> facts) {
        if (facts.isEmpty()) {
            return "You are a helpful AI assistant.";
        }

        String factList = facts.stream()
            .map(Document::getContent)
            .map(f -> "- " + f)
            .collect(Collectors.joining("\n"));

        return """
            You are a helpful AI assistant. Use this context about the user to personalise your responses:
            
            %s
            
            Don't explicitly reference that you know these things unless relevant. Let them inform your tone and examples naturally.
            """.formatted(factList);
    }
}

The async extraction is important. Users shouldn't wait for fact extraction to complete before getting a response. In a higher-scale system, push to a queue (SQS, Pub/Sub, Kafka) and process asynchronously.

Pattern 4: Conversation Compaction

When a long conversation exceeds what you want in the context window, you have two options: truncate (drop the oldest turns) or compact (summarise the older turns and keep the recent ones verbatim). Truncation is destructive. Compaction isn't.

@Service
public class ConversationCompactor {

    private final ChatClient compactionClient;

    private static final String COMPACTION_PROMPT = """
        Summarise this conversation segment into a compact context block.
        Preserve: all decisions made, all constraints mentioned, all specific numbers and identifiers.
        Drop: pleasantries, repetition, exploratory tangents that didn't lead anywhere.
        Output format: a single paragraph in past tense, max 200 words.
        """;

    private static final int COMPACTION_THRESHOLD = 30; // turns
    private static final int KEEP_RECENT = 10;          // turns to keep verbatim

    public List<Message> compact(List<Message> history) {
        if (history.size() < COMPACTION_THRESHOLD) {
            return history;
        }

        // Split: old segment to summarise + recent turns to keep
        List<Message> toSummarise = history.subList(0, history.size() - KEEP_RECENT);
        List<Message> recent = history.subList(history.size() - KEEP_RECENT, history.size());

        String segmentText = toSummarise.stream()
            .map(m -> m.getMessageType() + ": " + m.getContent())
            .collect(Collectors.joining("\n"));

        String summary = compactionClient.prompt()
            .system(COMPACTION_PROMPT)
            .user(segmentText)
            .call()
            .content();

        // Replace summarised segment with a single system message
        List<Message> compacted = new ArrayList<>();
        compacted.add(new SystemMessage("Earlier conversation summary: " + summary));
        compacted.addAll(recent);

        return compacted;
    }
}

The KEEP_RECENT = 10 is not arbitrary. The last 10 turns are the ones most likely to contain the context the user assumes you have right now. Summarising them risks losing the very thing they just told you. The further back you go, the safer it is to compress.

The Architecture That Combines Them

In a real production system, these patterns compose:

Each layer handles a different time horizon:

• Context window → this turn
• Redis → this session and recent sessions
• pgvector → lifetime knowledge about this user

The Traps I've Seen

Storing everything indiscriminately. Not every conversation contains a durable fact. If you run extraction on every turn of a customer service chat about a password reset, you'll pollute the semantic store with noise that degrades future retrieval. Run extraction selectively — after longer conversations, or only on turns where the model flagged something significant.

Forgetting the right to be forgotten. If users can request data deletion (GDPR, CCPA), your memory architecture needs a delete path. pgvector documents with userId metadata can be filtered and deleted. Redis keys with the user prefix can be scanned and dropped. If you can't answer "how do I delete all data for user X?" before launch, you have a compliance gap.

Memory that contradicts the current conversation. A user might tell you in October that they use Java 17, and in March tell you they've migrated to 21. The semantic store will have both facts. Without a recency weighting or a conflict resolution step, the model may use the stale one. One approach: include the extractedAt timestamp in the retrieved documents and instruct the model to prefer more recent facts when they conflict.

Not telling the user their data is being stored. Users have an intuition that an LLM forgets them. If you're storing memory, that's a meaningful departure from the default mental model. Be transparent about it — a "memory" indicator in the UI and a way to view or clear stored facts goes a long way toward trust.

Where to Start

If you're building a new AI application:

1. Start with in-context memory. Get the conversation flow right before adding persistence complexity.
2. Add Redis when you need cross-session continuity. One user, one deployment restart, one tab close shouldn't lose the conversation.
3. Add pgvector when you need personalization. When "knowing the user" would meaningfully change the quality of responses.
4. Add compaction when conversations routinely exceed 20–30 turns. Not before — it's complexity you don't need until you need it.

The pattern I described above took us about three weeks to implement properly, including the async extraction, the fact conflict handling, and the deletion endpoint. It reduced the "the AI forgot everything about me" complaints to essentially zero.

The LLM will always start each call from a clean slate. Your application layer is what decides whether that clean slate is actually blank — or whether it's pre-filled with everything the system should already know.

Build the layer. It's worth it.