Context Bottleneck
The set of architectural constraints created by the prompt window itself being a finite, expensive resource. As LLMs transition from stateless to stateful, the question of "what context to pack into the prompt" becomes the load-bearing engineering decision — but the prompt window doesn't scale linearly with usefulness. Beyond a certain length, additional context degrades reasoning quality (the "lost in the middle" problem), increases latency (the prefill tax), and burns through token budgets. The Context Bottleneck names this multi-axis tension between context-length, context-quality, and context-cost.
Definition
The set of architectural constraints created by the prompt window itself being a finite, expensive resource. As LLMs transition from stateless to stateful, the question of "what context to pack into the prompt" becomes the load-bearing engineering decision — but the prompt window doesn't scale linearly with usefulness. Beyond a certain length, additional context degrades reasoning quality (the "lost in the middle" problem), increases latency (the prefill tax), and burns through token budgets. The Context Bottleneck names this multi-axis tension between context-length, context-quality, and context-cost.