이미지: The Decoder 화면 갈무리
Summary
- A new study finds that session rules set by users disappear by an average of 83% when long conversations are summarized and compressed
- A Penn State research team measured this with an evaluation tool called COMPINT, finding that rule compliance dropped sharply after compaction compared to before
- The team raised retention to over 90% using a small add-on module based on Qwen3.5-9B
- 평균 지시 보존율
- 압축 후 17%(평균 83% 손실)
- 평가 도구
- 펜실베이니아주립대 개발 COMPINT
- 추가 모듈 베이스 모델
- Qwen3.5-9B
- 모듈 적용 후 보존율
- 에이전트 궤적 95.6%·장기 연구 95.1%·다중 턴 채팅 90.3%
- 비압축 상태 규칙 준수율
- 59~71%
- 특화 압축 프롬프트 보존율
- 40% 미만
- 공개 방식
- COMPINT와 추출 모듈 모두 GitHub 공개
The longer the conversation, the more AI forgets the rules
If you've ever told a chatbot something like "don't send emails without my approval," the longer that conversation runs, the less likely that rule is actually being followed. According to a study published by a Penn State research team, when AI systems summarize and compress long conversation histories, user-set rules disappear by an average of 83%. Only 17% survives.
Why compaction is necessary
AI models have a fixed limit on how much conversation they can read and remember at once — the context window. When users keep a conversation going in the same window instead of starting a new one, they eventually hit this limit. The industry's standard fix is "compaction": summarizing existing conversation content to free up space. The problem is that details inevitably get dropped during summarization.
The research team identified "session constraints" as the most vulnerable category of loss. These are rules that hold only until the conversation ends — things like "check with me before making changes" or "don't use my name in your responses." Because they aren't part of the task's core goals or next steps, they tend to fall outside the "task continuity" that compaction systems prioritize preserving. The team found that these rules are ignored after compaction in exactly this way.
How severe is it — measured with COMPINT
To quantify this loss, the team built an evaluation tool called COMPINT. Without compaction — when user rules remained fully intact — AI agents complied with rules at a rate of 59–71%. After compaction, this figure dropped sharply, often falling to levels comparable to having no rules at all. Most of the compaction systems tested performed worse than not compressing at all, with only GPT-5.4-mini exceeding the no-compaction baseline in some conditions.
Even when compaction prompts were specifically refined to better preserve user rules, there were limits. Even a retention-focused prompt the team designed separately failed to exceed a 40% retention rate.
The solution — a small module dedicated to extracting rules
The alternative the team proposed doesn't involve modifying the compaction system itself, but instead adding a small module that runs alongside it. This module is built on Qwen3.5-9B, a compact model from Alibaba's Qwen family. It reads all user input, identifies session constraints, compiles them into a separate list, and appends this list to the summary whenever the conversation is later compressed.
This extraction module achieved retention rates above 90% across all three tested scenarios: 95.6% in agent trajectory scenarios, 95.1% in long-horizon research tasks, and 90.3% in multi-turn chat. Notably, it requires no changes to the compaction system itself and no additional training.
| Scenario | Retention Rate After Applying Extraction Module |
|---|---|
| Agent trajectory | 95.6% |
| Long-horizon research tasks | 95.1% |
| Multi-turn chat | 90.3% |
Editor's take
What makes this study compelling isn't the problem itself, but where it hides. Users believe the rules they've set are being followed, but the moment a conversation grows long enough, that belief loses its footing. Worse, the failure is invisible: the AI keeps responding fluently, the task keeps moving forward, and the user has no idea compaction even happened. Then, at some point, an email goes out without approval, or information that was supposed to stay hidden gets exposed.
Anyone who has put agentic AI to real-world use will recognize this pattern. Rules set early in a conversation become less and less likely to be followed as that conversation grows longer. It's tempting to chalk this up to a model's "memory" limitations, but this study makes clear it's actually a design flaw in the compaction algorithm itself: a system built to prioritize task continuity structurally excludes the user's side conditions.
For organizations running chatbots or agents in long-lived sessions, the immediate practical step is to build a habit of repeatedly reasserting session rules rather than stating them once at the start. Rules that need to persist permanently should be embedded in the system prompt, kept separate from rules mentioned mid-conversation that are liable to be forgotten. Approval and security-related instructions in particular should be moved to system-level policy rather than left to rely on session memory alone.
It seems likely that within the next few months, retention modules like this one will become a default feature baked into the compaction capabilities of major AI platforms. Now that context compaction has become an industry standard, folding in an add-on that fixes its side effects looks like it's only a matter of time.
