benchrWhen it breaks
Halfway through, it stopped following the instructions I gave at the start
Also asked as the AI forgot my instructions · it ignored the style guide I pasted · it went back to bullet points · it forgot the character's name
Cause documented by a vendor or a published paper Not tested by benchr First logged Last checked
What is actually happening
Nothing was forgotten in the human sense. Either the earlier turns fell out of the window, or they are still there and losing to more recent, more specific text.
Why
- A conversation has a fixed token window. When it fills, the oldest turns stop being sent.
- Even inside the window, instructions compete. A long document pasted after your rules gives the model far more recent evidence about what you want than the rules do.
- Vendors publish the window size but not how attention degrades across it, so the point where adherence starts to slip is undocumented.
The quick fix
Re-state the two or three rules that matter immediately before the request, not at the top of the conversation. Recency beats position.
The real fix
Move standing rules out of the conversation into the system prompt or a project instruction, where they are re-sent on every turn instead of aging out. Then keep the conversation short and start a new one per task.
Step by step
- Put non-negotiable rules in the system prompt / custom instructions / project settings.
- Keep each conversation to one task. A fresh thread costs nothing and resets the competition.
- When pasting a long reference, put your instruction after it, not before.
- Ask it to restate the constraints in its own words before it starts — a cheap check that they landed.
- For long jobs, have it write the constraints to a file or a message it re-reads, rather than relying on the thread.
If you are seeing an API error
Grounded in
- Fit a whole codebase in one prompt, and find its limitThe window sizes are documented; how adherence degrades across them is not, and that gap is recorded on the capability itself.