The Treachery of Memory: On Long Contexts and Agentic Failures
Long context is your friend… when we are talking about summarization and retrieval. For agentic workflow it is often detrimental due to reasons such as:
Context poisoning, where the model hallucinates and messes with its own context. The ripples are powerful and they die slowly;
Context distraction, where the model starts to repeat itself instead of trying new strategies;
Context confusion, which happens when one gives the model too many tools (sometimes 2 is too many);