LangChain Memory: Giving Your Application Context

This is the third post in our LangChain fundamentals series, building on chains and RAG applications.

We covered why an ever-growing conversation history leads to linearly-growing cost, and covered sliding windows and summarization as the two standard fixes, in our OpenAI series. LangChain's memory abstractions are essentially structured, reusable implementations of exactly those same patterns, wired directly into a chain rather than managed by hand.

Message history: the baseline

from langchain_core.chat_history import InMemoryChatMessageHistory
from langchain_core.runnables.history import RunnableWithMessageHistory
 
store = {}
 
def get_session_history(session_id: str):
    if session_id not in store:
        store[session_id] = InMemoryChatMessageHistory()
    return store[session_id]
 
chain_with_history = RunnableWithMessageHistory(
    rag_chain,  # from the previous post in this series
    get_session_history,
    input_messages_key="question",
    history_messages_key="history",
)
 
response = chain_with_history.invoke(
    {"question": "How many vacation days do I get?"},
    config={"configurable": {"session_id": "user-123"}},
)

RunnableWithMessageHistory wraps any chain (here, the RAG chain built in the previous post) with automatic conversation tracking, keyed by session_id — meaning a real application serving many concurrent users keeps each user's conversation genuinely separate, without manually threading a conversation ID through every function call by hand.

The cost problem doesn't disappear — it's just managed more consistently

This is the detail worth being explicit about: InMemoryChatMessageHistory is functionally the same "append every message forever" pattern flagged as a real cost problem in our OpenAI series — LangChain gives it a clean interface, but the underlying growth-without-bound issue is identical unless you deliberately manage it.

from langchain.memory import ConversationSummaryBufferMemory
 
memory = ConversationSummaryBufferMemory(
    llm=model,
    max_token_limit=500,
)

ConversationSummaryBufferMemory is the direct, built-in equivalent of the manual summarization function from our OpenAI cost-management post: it keeps recent messages verbatim up to a token budget, and automatically summarizes older messages using the model itself once that budget is exceeded — the same trade-off (full fidelity for recent context, compressed fidelity for older context), implemented once, reusable across every chain that needs it, instead of hand-written per application.

Trimming messages explicitly, when you want direct control

Beyond the summarizing memory class, LangChain also exposes a lower-level trimming utility for cases where you want direct, explicit control over exactly how history is bounded, rather than relying on a memory class's built-in policy:

from langchain_core.messages import trim_messages
 
trimmed = trim_messages(
    messages,
    max_tokens=1000,
    strategy="last",
    token_counter=model,
    include_system=True,
)

strategy="last" keeps the most recent messages up to the token budget, always preserving the system message (include_system=True) regardless of how far back it sits in the list — worth reaching for specifically when a chain's history-handling needs differ from what ConversationSummaryBufferMemory's summarize-then-truncate policy provides, or when you want the trimming logic itself to be transparent and directly inspectable rather than delegated to a memory class.

Persisting history beyond a single process

InMemoryChatMessageHistory disappears the moment the process restarts — genuinely fine for local development, a real problem for anything running in production across multiple server instances or surviving a redeploy.

from langchain_community.chat_message_histories import RedisChatMessageHistory
 
def get_session_history(session_id: str):
    return RedisChatMessageHistory(session_id=session_id, url="redis://localhost:6379")

Swapping the in-memory store for a Redis-backed one is, again, a one-line change to the same interface — conversation history now survives a process restart and is shared correctly across multiple server instances handling the same user's requests, which matters the moment a real application runs behind a load balancer with more than one backend instance, the exact networking pattern covered earlier in this blog for Docker and Kubernetes.

Memory in a RAG context: a subtlety worth naming

Combining memory with RAG (as the example at the top of this post does) introduces a real design question: should a follow-up question be rephrased using conversation history before retrieval runs, so retrieval itself receives full context?

condense_prompt = ChatPromptTemplate.from_messages([
    ("system", "Rephrase the follow-up question as a standalone question, using the chat history for context."),
    ("user", "Chat history:\n{history}\n\nFollow-up question: {question}"),
])

Without this step, a user asking "what about sick days?" as a follow-up to "how many vacation days do I get?" sends only "what about sick days?" to the retriever — which has no idea that question is actually about a company's paid-time-off policy, and may retrieve entirely unrelated content. Condensing the follow-up into a standalone question first ("What is the company's policy on sick days?") gives the retrieval step what it actually needs to find the right passage. This is a genuinely common, easy-to-miss gap in a first RAG-plus-memory implementation.

Vector-store-backed memory: retrieving relevant history, not just recent history

A sliding window or summary buffer both assume "recent" is the same as "relevant" — usually true, but not always. A long-running conversation that returns to a topic discussed thirty turns ago benefits from retrieving that specific earlier exchange, not just whatever happened most recently:

from langchain.memory import VectorStoreRetrieverMemory
 
retriever = vectorstore.as_retriever(search_kwargs={"k": 3})
memory = VectorStoreRetrieverMemory(retriever=retriever)
 
memory.save_context(
    {"input": "What's our policy on remote work?"},
    {"output": "Employees can work remotely up to 3 days per week."},
)
 
relevant_history = memory.load_memory_variables({"prompt": "Can I work from home more?"})

This treats past conversation turns the same way a RAG pipeline treats documents — embedded, stored, and retrieved by semantic similarity to the current message, rather than kept purely by recency. It's a genuinely different trade-off than a sliding window: more expensive to maintain (every turn needs embedding and storage), and better suited to long-lived, topic-varied conversations where "what was relevant three days ago" isn't reliably "what was said most recently."

Choosing the right memory strategy for the actual conversation shape

• Short, single-session conversations (a support chat that resolves in one sitting): a sliding window is usually sufficient — recent context is genuinely almost always the relevant context.
• Long conversations with real topic drift: ConversationSummaryBufferMemory keeps the gist of everything while bounding cost, at the price of losing some verbatim detail from older turns.
• Long-lived, multi-session relationships (a personal assistant used over weeks): vector-store-backed memory is worth the added complexity, since "relevant" and "recent" genuinely diverge once a conversation spans that much time and topic variety.

Defaulting to the simplest option (a sliding window) and only reaching for something more sophisticated once a real conversation pattern demonstrates the need is the same "don't add complexity you can't yet justify" instinct that applies to reaching for LangGraph over a plain chain, covered in the next series on this blog.

Clearing a session's memory explicitly

A conversation occasionally needs a deliberate reset — a user asking to "start over," or a support session that's genuinely concluded and shouldn't influence a later, unrelated one:

store["user-123"].clear()

Exposing this as an explicit action in an application, rather than only ever accumulating history until it expires or gets summarized away, matters for any use case where a stale earlier context could actually mislead a later, unrelated conversation with the same user.

Inspecting what's actually stored, for debugging

When a chain's answer suggests it's missing context it should have, checking the raw stored history directly — rather than guessing — confirms whether the problem is missing memory or something else entirely:

print(store["user-123"].messages)

What to actually remember from this post

• RunnableWithMessageHistory tracks conversation state per session, wrapping any existing chain without rewriting it.
• The underlying cost problem from our OpenAI series doesn't disappear — ConversationSummaryBufferMemory is the built-in, reusable equivalent of the manual summarization fix covered there.
• trim_messages gives explicit, transparent control over history bounding, when a memory class's built-in policy isn't the right fit.
• Swap the history backend (Redis, a database) for anything that needs to survive a process restart or run correctly across multiple server instances.
• Condense follow-up questions into standalone ones before retrieval, in a RAG-plus-memory setup — otherwise the retriever never sees the context a follow-up question actually depends on.

Next in the series: LangChain Agents and Tools Explained, where a chain stops following one fixed sequence and starts deciding its own next step.