Chapter 4

The context window has blind spots in the middle

When software vendors demonstrate modern language models to financial institutions, their favorite specification is context length.

They announce windows of one hundred thousand, one million, or even two million tokens. The sales pitch is intoxicating: Do not waste time pre-filtering documents. Simply drop entire 10-K filings, debt agreements, and earnings transcripts into the prompt at once. The model will read everything.

To an analyst facing hundreds of pages of dense financial disclosures, this sounds like salvation.

It is an expensive illusion.

The context window is the maximum number of tokens a model can accept in a single request. But having a million-token capacity does not mean the model reads each token with equal sharpness. In practice, long prompts introduce severe cognitive blind spots, flatten financial tables, and multiply computational costs.

Attention forms a U-shaped valley across long prompts

How does a language model process a document containing fifty thousand tokens?

It uses a mathematical mechanism called attention. Attention calculates how strongly every single token in the prompt relates to every other token. When a prompt is short, the model connects terms effortlessly.

When a prompt spans eighty pages, attention begins to degrade.

Extensive empirical research reveals a consistent pattern in neural networks: the lost-in-the-middle effect. Models pay intense attention to tokens at the very beginning of the prompt (the opening instructions) and tokens at the very end of the prompt (the most recent text before generation).

In the middle of the document, attention sags into a deep valley.

Attention is strong at the edges but sags into a valley in the middle of long documents

If you paste an entire 10-K report into a prompt, the opening business summary and the concluding audit signature sit in prime attention zones.

Where do debt covenant restrictions (clauses in credit agreements requiring borrowers to operate within strict financial limits), non-standard lease obligations, and contingent legal liabilities live?

They sit right in the middle: buried in Note 8, Note 14, and Note 19 on pages 45 through 75.

When an analyst asks a question about restrictive debt ratios, the model often glides past the buried qualification. It does not crash or raise an error. Instead, it generates a fluent, confident answer based on general corporate boilerplate from the top of the filing, completely missing the restrictive covenant.

Flattening multi-column tables scrambles rows and detaches footnotes

The second major hazard of long context dumping is table flattening.

Financial filings are not continuous prose. More than half the analytical value of an annual report resides in two-dimensional financial statements: balance sheets, cash flow schedules, and segmented income tables.

A language model does not see a visual grid of columns and rows. It ingests a flat, linear stream of sub-word tokens reading from left to right.

When your software extracts text from a PDF filing and passes it into the context window, it must serialize that visual grid into sequential text. In that conversion, three things routinely break:

  1. Shifted columns: An empty cell or an unaligned sub-total causes numbers to slide into adjacent columns. The model reads an operating expense figure as a revenue line.
  2. Displaced parentheses: In accounting, parentheses denote negative numbers or cash outflows (such as capital expenditures). In linear serialization, parentheses often dissociate from numbers, turning a $300 million cash outflow into an apparent cash inflow.
  3. Detached footnotes: Financial tables rely heavily on footnote markers ("(1) Excludes restructuring charges of $45 million"). In a flattened text stream, that small footnote marker detaches from the operating income row and floats at the bottom of the page, invisible to the model's extraction logic.

If you dump raw tabular pages into a prompt, you are not testing the model's intelligence; you are testing whether its linear attention can re-construct a scrambled grid.

Running scenario: The specimen covenant hides in Note 8

Let us observe the lost-in-the-middle effect on Acme Industrial Technologies.

(Note for live testing: To witness this failure mode in your own chat interface, you must paste the full text of an actual thirty-to-forty-page SEC filing. A short snippet will not trigger attention decay. The block below is an abbreviated schematic illustrating how Note 8 gets buried between opening financial statements and closing auditor certifications.)

Instead of pasting a clean, five-line excerpt, you paste a long filing package into the prompt box. The text opens with Item 1 (Business Overview) and Item 8 (Consolidated Financial Statements, displaying our specimen fourth-quarter revenue figure of $4,280 million), followed by twenty pages of standard accounting policies. Buried deep in Note 8 on page 28 sits the critical credit agreement covenant, followed by ten more pages of legal certifications and auditor sign-offs.

You submit your prompt to the model:

Document:
[Item 8: Consolidated Financial Statements
Acme Industrial Technologies reported fourth-quarter net sales of $4,280 million, representing 6% growth year-over-year. Operating income reached $640 million.

... [Twenty pages of general corporate accounting policies, consolidation guidelines, depreciation schedules, and employee benefit disclosures] ...

Note 8: Debt Covenants and Credit Facilities
Under the Senior Credit Facility dated November 14, consolidated net funded debt (interest-bearing debt less cash and equivalents) to EBITDA (earnings before interest, taxes, depreciation, and amortization) shall not exceed 3.50 to 1.00 at the close of any fiscal quarter. Consolidated funded debt stood at $2,800 million against cash and equivalents of $360 million. Consolidated EBITDA for the trailing twelve months reached $1,220 million. Failure to maintain this ratio constitutes an immediate event of default.

... [Ten pages of subsequent event disclosures, internal control certifications, and independent auditor attestations] ...
Item 9A: Controls and Procedures
Management assessed the effectiveness of internal control over financial reporting as of December 31, concluding that controls were effective under COSO guidelines.]

Instructions:
Identify the restrictive debt covenant governing Acme Industrial Technologies.
State the exact maximum leverage ratio allowed under the Senior Credit Facility.
Use two concise sentences.

The model reads through forty thousand tokens of text and outputs its answer:

Acme Industrial Technologies maintains senior credit facilities to support general working capital and liquidity requirements. The company reports ongoing compliance with all customary financial covenants, with no events of default recorded during the fiscal year.

The response is grammatically polished and broadly true, but it failed the analytical test.

It never stated the 3.50 to 1.00 ratio. While our specimen revenue figure of $4,280 million sat clearly in the opening income statement tables, the specific covenant restriction was buried in the attention valley of Note 8 on page 28. The model picked up the general liquidity boilerplate from the opening pages and generated a vague confirmation of compliance.

In finance, missing a restrictive debt covenant is not a stylistic flaw. If Acme's leverage ratio rises to 3.60x, that missed sentence triggers a technical default: an immediate contractual breach permitting lenders to demand accelerated debt repayment.

How do you ground model responses in verified documents without overwhelming the context window?

How can an institution give the model access to two hundred pages of filings without dumping the whole file into the prompt? How do you isolate the exact paragraph in Note 8 so the model reads it with sharp attention?

The Leader's View: Structure and isolate tabular inputs rather than dumping raw filings

When building enterprise generative AI capabilities, leadership must push back against the temptation of mega-context dumping.

Treating the context window as an infinite document dump creates an illusion of thoroughness while maximizing cost and omission risk.

Operating rules to mandate

  1. Never dump unparsed 10-K PDFs into a single prompt: Require workflows to extract and parse specific sections (such as MD&A, the management discussion section explaining operating results, Note 8, or the Consolidated Balance Sheet) into discrete, targeted prompts.
  2. Mandate tabular preservation engines: Before financial statements enter a model prompt, run them through dedicated structural parsers that preserve cell coordinates, row hierarchies, and footnote attachments in clean structured formats (such as markdown tables or JSON).
  3. Audit omissions as material control failures: In financial research and risk underwriting, an omission is far more hazardous than a stylistic typo. Measure vendor models specifically on their retrieval recall rate for middle-document disclosures.

Questions to put to a vendor

  1. What is your system's measured retrieval accuracy on facts located in the 40% to 60% depth range of a 100,000-token prompt?
  2. How does your document ingestion pipeline convert multi-column financial tables, and how does it guarantee that footnote caveats remain bound to numerical cells?

Zoom-out (Historic): From microfiche archives to linear context streams

The struggle to navigate dense corporate disclosure filings is not a novel challenge created by modern language models; it is the continuation of a century-long struggle over financial archival retrieval.

Before the digitization of securities markets in the 1990s, an equity analyst conducting deep historical research did not query a computer terminal. They traveled to financial libraries or broker archives to pull physical microfiche cards: photographic sheets containing miniaturized images of corporate balance sheets, quarterly statements, and bond indentures. The analyst manually aligned the microfiche card under a mechanical optical reader, adjusting manual focus dials to inspect footnote covenants one page at a time.

In 1993, regulatory authorities modernized corporate filing through the Electronic Data Gathering, Analysis, and Retrieval (EDGAR) system.

For the first time, public annual and quarterly reports were transmitted as text files over computer networks. Yet EDGAR solved distribution, not comprehension: an analyst still had to download three-hundred-page plain-text filings, manually searching for footnote caveats and covenant terms.

Large language models represent the latest evolution in this archival lineage. But as we observed with the lost-in-the-middle effect, feeding an entire filing into a flat token stream recreates the exact fatigue of the microfiche reader: the key footnote covenant remains buried in a sea of text. For institutions, the solution has never been dumping the entire archive on the analyst's desk; the true solution has always been disciplined indexing and targeted retrieval.