Apple Improves Context Window Management for its Foundation Models

"The context window is a critical resource used to hold system instructions, user prompts, and model responses. Apple's Foundation Models run on-device, offering a relatively small context window that can fill up quickly."

"To recover from the .exceededContextWindowSize error, developers need to start a new session and reinitialize its state, ensuring the existing workflow continues without impairing user experience."

"iOS 26.4 introduced a new contextSize property on SystemLanguageModel, which returns the available context capacity, along with a tokenCount(for:) method to measure how many tokens a given input consumes."

iOS 26.4 introduces improved context window management for Apple's Foundation Models, addressing the 4096-token limit. Developers must manage this resource actively, especially in chat-like sessions where prompts and responses accumulate. An .exceededContextWindowSize error occurs when the limit is reached, requiring session reinitialization. Apple provides strategies for developers, including task splitting and prompt trimming. The new contextSize property and tokenCount(for:) method help track context usage and token consumption, allowing for dynamic adaptation without hardcoding limits.

Read at InfoQ