Chapter 4

The context window has blind spots in the middle

When software vendors demonstrate modern language models to financial institutions, their favorite specification is context length.

They announce windows of one hundred thousand, one million, or even two million tokens. The sales pitch is intoxicating: Do not waste time pre-filtering documents. Simply drop entire 10-K filings, debt agreements, and earnings transcripts into the prompt at once. The model will read everything.

To an analyst facing hundreds of pages of dense financial disclosures, this sounds like salvation.

It is an expensive illusion.

The context window is the maximum number of tokens a model can accept in a single request. But having a million-token capacity does not mean the model reads each token with equal sharpness. In practice, long prompts introduce severe cognitive blind spots, flatten financial tables, and multiply computational costs.

Attention forms a U-shaped valley across long prompts

How does a language model process a document containing fifty thousand tokens?

It uses a mathematical mechanism called attention. Attention calculates how strongly every single token in the prompt relates to every other token. When a prompt is short, the model connects terms effortlessly.

When a prompt spans eighty pages, attention begins to degrade.

Extensive empirical research reveals a consistent pattern in neural networks: the lost-in-the-middle effect. Models pay intense attention to tokens at the very beginning of the prompt (the opening instructions) and tokens at the very end of the prompt (the most recent text before generation).

In the middle of the document, attention sags into a deep valley.

Attention is strong at the edges but sags into a valley in the middle of long documents

If you paste an entire 10-K report into a prompt, the opening business summary and the concluding audit signature sit in prime attention zones.

Where do debt covenant restrictions, non-standard lease obligations, and contingent legal liabilities live?

They sit right in the middle: buried in Note 8, Note 14, and Note 19 on pages 45 through 75.

When an analyst asks a question about restrictive debt ratios, the model often glides past the buried qualification. It does not crash or raise an error. Instead, it generates a fluent, confident answer based on general corporate boilerplate from the top of the filing, completely missing the restrictive covenant.

Flattening multi-column tables scrambles rows and detaches footnotes

The second major hazard of long context dumping is table flattening.

Financial filings are not continuous prose. More than half the analytical value of an annual report resides in two-dimensional financial statements: balance sheets, cash flow schedules, and segmented income tables.

A language model does not see a visual grid of columns and rows. It ingests a flat, linear stream of sub-word tokens reading from left to right.

When your software extracts text from a PDF filing and passes it into the context window, it must serialize that visual grid into sequential text. In that conversion, three things routinely break:

  1. Shifted columns: An empty cell or an unaligned sub-total causes numbers to slide into adjacent columns. The model reads an operating expense figure as a revenue line.
  2. Displaced parentheses: In accounting, parentheses denote negative numbers or cash outflows (such as capital expenditures). In linear serialization, parentheses often dissociate from numbers, turning a $300 million cash outflow into an apparent cash inflow.
  3. Detached footnotes: Financial tables rely heavily on footnote markers ("(1) Excludes restructuring charges of $45 million"). In a flattened text stream, that small footnote marker detaches from the operating income row and floats at the bottom of the page, invisible to the model's extraction logic.

If you dump raw tabular pages into a prompt, you are not testing the model's intelligence; you are testing whether its linear attention can re-construct a scrambled grid.

Running scenario: The specimen covenant hides in Note 8

Let us observe the lost-in-the-middle effect on Acme Industrial Technologies.

Instead of pasting a clean, five-line excerpt, you paste a massive prompt containing forty pages of Acme's annual report. The text begins with Item 1 (Business Overview) and Item 7 (Management's Discussion), continues into Item 8 containing the Consolidated Statements of Earnings (displaying our specimen fourth-quarter revenue figure of $4,280 million), details thirty pages of disclosure notes, and concludes with Item 9.

Buried on page 28, deep in Note 8 (Debt Covenants and Credit Facilities), sits this exact sentence:

"Under the Senior Credit Facility dated November 14, consolidated funded debt to EBITDA shall not exceed 3.50 to 1.00 at the close of any fiscal quarter."

You submit your prompt to the model:

Document:
[Forty continuous pages of Acme Industrial Technologies 10-K filing text, spanning Item 1 through Item 8, including Consolidated Statements of Earnings reporting fourth-quarter net sales of $4,280 million, through Note 8 debt disclosures...]

Instructions:
Identify the restrictive debt covenant governing Acme Industrial Technologies.
State the exact maximum leverage ratio allowed under the Senior Credit Facility.
Use two concise sentences.

The model reads through forty thousand tokens of text and outputs its answer:

Acme Industrial Technologies maintains senior credit facilities to support general working capital and liquidity requirements. The company reports ongoing compliance with all customary financial covenants, with no events of default recorded during the fiscal year.

The response is grammatically polished and broadly true, but it failed the analytical test.

It never stated the 3.50 to 1.00 ratio. While our specimen revenue figure of $4,280 million sat clearly in the opening income statement tables, the specific covenant restriction was buried in the attention valley of Note 8 on page 28. The model picked up the general liquidity boilerplate from the opening pages and generated a vague confirmation of compliance.

In finance, missing a restrictive debt covenant is not a stylistic flaw. If Acme's leverage ratio rises to 3.60x, that missed sentence triggers a technical default.

How do you ground model responses in verified documents without overwhelming the context window?

Here is the problem to think about before you move to the next chapter: how can an institution give the model access to two hundred pages of filings without dumping the whole file into the prompt? How do you isolate the exact paragraph in Note 8 so the model reads it with sharp attention?

The Leader's View: Structure and isolate tabular inputs rather than dumping raw filings

When building enterprise generative AI capabilities, leadership must push back against the temptation of mega-context dumping.

Treating the context window as an infinite document dump creates an illusion of thoroughness while maximizing cost and omission risk.

Operating rules to mandate

  1. Never dump unparsed 10-K PDFs into a single prompt: Require workflows to extract and parse specific sections (such as MD&A, Note 8, or the Consolidated Balance Sheet) into discrete, targeted prompts.
  2. Mandate tabular preservation engines: Before financial statements enter a model prompt, run them through dedicated structural parsers that preserve cell coordinates, row hierarchies, and footnote attachments in clean structured formats (such as markdown tables or JSON).
  3. Audit omissions as material control failures: In financial research and risk underwriting, an omission is far more hazardous than a stylistic typo. Measure vendor models specifically on their retrieval recall rate for middle-document disclosures.

Questions to put to a vendor

  1. What is your system's measured retrieval accuracy on facts located in the 40% to 60% depth range of a 100,000-token prompt?
  2. How does your document ingestion pipeline convert multi-column financial tables, and how does it guarantee that footnote caveats remain bound to numerical cells?

Zoom-out: Compare massive context windows with structured enterprise search

The struggle over context length illuminates a broader architectural debate across enterprise artificial intelligence: brute-force context ingestion versus structured search.

In the consumer AI space, the prevailing trend is brute force. Model providers expand context windows to millions of tokens, encouraging users to upload entire books and video files. The underlying marketing narrative implies that search engines and databases are obsolete; the neural network will simply absorb the entire corporate archive.

In regulated institutional finance, brute force fails three critical operational tests:

First, auditability. When an institution defends a financial filing or valuation memo before regulators or an audit committee, every fact must carry an unbroken chain of custody. In a structured retrieval system, the software logs the exact document hash, page number, and paragraph cited. In a million-token prompt, the lineage disappears into an opaque mathematical blur.

Second, security. Brute-force context ingestion bypasses enterprise permissions. If a system loads an entire corporate directory into a prompt, any user querying the model can inadvertently surface confidential compensation figures or unannounced M&A plans. Structured retrieval enforces access control before a single word reaches the model.

Third, operating economics. Re-transmitting hundreds of pages on every query costs dollars per turn and introduces substantial network latency. Structured retrieval extracts fifty lines of verified text, costs fractions of a cent, and responds in seconds. For regulated enterprises, the future of artificial intelligence is not bigger prompt dumps; it is disciplined, surgical retrieval.