Ask a large language model to summarize a quarter's financial performance and it will produce fluent, confident, well-structured prose within seconds, citing specific figures, calculating margins, and drawing comparisons to prior periods. The prose will read exactly as well whether every number in it is correct or whether several are quietly wrong. That single fact, more than any prompt engineering technique, is the one a business needs to internalize before letting AI anywhere near a financial narrative that anyone will actually rely on.
Key Takeaway
A 2025 benchmark called FAITH, evaluating large language models on financial tabular data extraction, found leading models collapse from 95.6% accuracy on simple, single-value lookups to near-zero accuracy on multivariate calculations requiring multiple steps. This is not a prompting failure correctable through better instructions; it reflects a structural limitation in how these models generate text, predicting plausible next tokens rather than executing deterministic computation. Prompt engineering meaningfully improves consistency, format, and the model's ability to stay grounded in provided source material. It does not, and by the architecture's own nature cannot, make the underlying arithmetic reliable. The businesses using AI safely for financial narrative are the ones that have separated these two facts operationally, not just understood them intellectually.
The Benchmark Number That Should Change How You Use This
FAITH, "A Framework for Assessing Intrinsic Tabular Hallucinations in Finance," is a 2025 benchmark specifically designed to quantify how LLM accuracy degrades as financial information-extraction tasks increase in complexity[1]. The headline finding, cited in subsequent 2026 research on financial statement verification, is stark: top-performing models that achieve roughly 95.6% accuracy on simple lookups, correctly pulling a single stated figure from a table, collapse to accuracy approaching zero when the task requires a multivariate calculation, combining multiple figures across several steps to reach a derived number[2].
This is not a marginal degradation. It is a near-total collapse in reliability precisely at the point where a task stops being "find this number" and becomes "compute this number from several others," which describes an enormous share of what an actual financial narrative requires: a gross margin, a period-over-period variance, a weighted average, a ratio. A model that reliably retrieves a single revenue figure and unreliably computes the margin derived from that figure and a corresponding cost figure will produce a narrative in which the retrieved facts are correct and the calculated conclusions drawn from them frequently are not, with no stylistic difference in confidence or fluency between the two.
Why LLMs Are Bad At Arithmetic By Design
The mechanism behind this is worth understanding precisely, because it explains why the problem is structural rather than a matter of insufficiently careful prompting. A large language model, at its core, is trained to predict the statistically most plausible next token in a sequence, based on patterns learned across enormous volumes of text. This architecture is extraordinarily effective at producing fluent, contextually appropriate language, and reasonably effective at tasks that resemble patterns heavily represented in training data. It is not, by design, executing deterministic arithmetic the way a calculator or spreadsheet formula does; when a model "computes" a sum, it is generating the token sequence most statistically consistent with what a correct-looking answer to that kind of question tends to look like, which is a fundamentally different process than actually performing the calculation, and one that degrades predictably as the calculation's complexity moves further from patterns well-represented in training data[3].
Recent technical research has proposed neuro-symbolic approaches specifically to address this, arguing that language models should be "surgically" relieved of arithmetic responsibility entirely, deferring any actual computation to a deterministic external tool or fact ledger while the model handles only the language generation around a result computed elsewhere[4]. This is a meaningful technical direction, and tools built on this principle, where the AI drafts narrative but a separate, deterministic calculation engine produces every number, are measurably safer than tools where the same model does both. It also confirms, from the technical solution's own design logic, that the underlying problem is not something better prompting resolves; it is something the architecture needs to be structurally routed around.
Memorization, Not Reasoning
A further, related finding sharpens the concern beyond simple arithmetic accuracy. Research examining how LLMs perform on financial prediction and analysis tasks when the underlying statements are deliberately perturbed, altered in ways that would change a correct answer while preserving surface-level similarity to familiar patterns, found that model accuracy on these perturbed statements dropped to what would be expected from random chance[5]. The implication is significant: strong performance on standard, unperturbed financial analysis tasks may substantially reflect the model recognizing and reproducing patterns similar to material in its training data, rather than performing genuine, generalizable financial reasoning from first principles. A model can appear highly competent at analyzing familiar-looking financial statements while its actual reasoning capability, tested against statements engineered to look familiar but require a different correct answer, is barely better than guessing.
The Narrative Risk Is Different From The Number Risk
This is the specific point this article's title points at, and it deserves to be stated as directly as the research supports. Industry analysis of AI hallucination risk in financial reporting has identified narrative-heavy sections, management commentary, footnote disclosures, and earnings-release language, as the highest-risk application of AI in financial reporting, and the reason is not that these sections involve harder arithmetic[6]. It is that fluent, well-structured narrative prose is precisely the context in which a human reviewer is most likely to read quickly, trust the overall coherence of the writing, and under-scrutinize the specific figures embedded within it. A spreadsheet formula error is visually conspicuous once you look for it. A hallucinated figure embedded in a well-written paragraph, stated with exactly the same confident register as every accurate figure around it, provides no visual or stylistic signal that it deserves extra scrutiny, which is precisely why it survives review more often than a comparable spreadsheet error would.
The Retrieval Trap
A common, reasonable-sounding mitigation deserves specific scrutiny because it is less protective than it appears. Retrieval-augmented generation, where a model is given access to a business's own source documents to ground its answers in verified facts before generating a response, is widely marketed as the fix for hallucination, on the theory that a model reasoning from your actual data cannot invent figures that aren't there. Industry benchmarking published in 2026, however, found that even a strong model paired with an imperfect retrieval layer inherits the retrieval layer's own errors, and that the retrieval step itself, correctly identifying and surfacing the exact relevant data from a larger document set, is frequently the weaker link in the pipeline rather than the generation step that follows it[6]. Grounding a model in your own documents reduces one category of error, invented figures with no basis in any source, while leaving a second category largely intact: figures correctly sourced from the right document but incorrectly calculated, combined, or contextualized once retrieved.
What Prompt Engineering Can And Cannot Fix
Given the mechanism described above, it is possible to state with reasonable precision what careful prompt design actually improves and what it does not, which is the central, practically useful distinction this article argues for.
What it fixes: Format and structural consistency across repeated outputs; explicit instruction to cite the specific source of every stated figure, which does not prevent a wrong calculation but does make a wrong calculation easier for a human reviewer to trace and verify; constraining the model's tone and register to avoid overstating confidence in genuinely uncertain conclusions; and reducing the incidence of fully invented figures with no grounding in provided source material, particularly when combined with retrieval grounding.
What it does not fix: The underlying reliability of any multi-step calculation the model performs internally, which remains governed by the architectural limitation described above regardless of how the prompt is worded; the risk that a model reasons well on familiar-looking data through pattern-matching while failing on data that differs from common patterns in ways a prompt cannot detect or correct for; and the fundamental fact that fluent, well-formatted prose provides no reliable signal, to the model or to a rushed human reviewer, about which specific figures within it are actually correct.
A Canadian Bank Said So, In Writing
It is worth noting that this is not merely academic caution from outside the financial industry; a Canadian financial institution has published its own technical research reaching the same conclusion. A 2026 paper from CIBC researchers, "Hallucination Detection and Mitigation in Large Language Models," introduces an operational framework for managing hallucination risk in high-stakes domains including finance, built explicitly on the premise that hallucination is a persistent, structural characteristic of these models requiring continuous, systematic management rather than a bug to be prompted away[7]. Their framework categorizes hallucination sources into model, data, and context-related factors, and pairs detection methods, including uncertainty estimation and reasoning-consistency checks, with staged mitigation strategies. The detail worth emphasizing for a Canadian business audience is who published this: a major domestic bank's own research team, working on exactly the reliability problem this article describes, treating it as an ongoing operational risk-management discipline rather than a solved problem.
A Practical Protocol For Using AI On Financial Narrative
Drawing directly from the mechanism described above, a specific, workable protocol follows for any business using AI to draft financial commentary, board materials, or client-facing financial narrative.
Never let the model compute a number it will also narrate. Provide every figure the narrative will reference as verified input, calculated separately in your accounting system or a deterministic spreadsheet, and instruct the model explicitly to use only the figures provided rather than deriving new ones. This single practice addresses the core finding of the FAITH benchmark directly, by removing the multivariate calculation step from the model's responsibility entirely.
Require inline sourcing for every figure. A prompt instruction to cite the specific source line for each number used does not prevent an error, but it converts an invisible error into a traceable one, since a reviewer can check the citation against the source in seconds rather than needing to re-derive the figure independently.
Treat narrative fluency as evidence of nothing. Build review habits that explicitly counteract the tendency to under-scrutinize well-written prose; a specific, useful practice is having a reviewer check every number in a draft against source data before reading the surrounding narrative at all, so the fluency of the writing cannot anchor their assessment of the numbers embedded in it.
Test the tool against a known-perturbed case before trusting it on a live one. Given the memorization-versus-reasoning finding above, periodically testing an AI narrative tool against a deliberately modified version of a familiar report, where you know the correct answer differs from the familiar pattern, is a more honest test of its actual reasoning reliability than testing it only on standard, expected data.
The Regulatory Backdrop
The regulatory environment around this question is asymmetric across jurisdictions in a way Canadian businesses using AI tools should be aware of even absent domestic AI-specific law. The European Union's AI Act imposes binding obligations, human oversight, traceability, transparency, and event logging, on high-risk AI systems, with a key compliance milestone landing August 2, 2026, and penalties reaching into the tens of millions of euros or a percentage of global turnover for serious breaches[8]. As discussed elsewhere in this series, Canada currently has no equivalent binding federal statute; the Artificial Intelligence and Data Act died on the order paper in January 2025. A Canadian business using AI tools built by, or integrated with, vendors exposed to EU obligations may find itself operating under EU-influenced product design and documentation standards indirectly, even though no equivalent obligation currently binds the business itself under Canadian law, which is a genuinely unusual position: meaningfully regulated by proxy, through vendor compliance, while unregulated directly.
The Confidence Calibration Problem
A further distinction worth drawing sharpens why this problem is specifically dangerous in financial narrative rather than in other AI writing applications. Language models are not, by default, well calibrated to express uncertainty proportional to their actual reliability on a given task; a model asked to state a calculated margin will typically state it with the same declarative confidence whether the underlying computation was a simple, reliable lookup or a complex, error-prone multi-step derivation, because nothing in the generation process tracks or surfaces that distinction to the reader. Research on uncertainty estimation in language models has treated this calibration gap as a distinct, actively studied problem, separate from the accuracy problem itself, precisely because a model that is wrong but appropriately hedged behaves very differently, from a risk-management standpoint, than one that is wrong and stated with full declarative confidence[7].
This means asking a model to "flag your confidence level" in a prompt, a common and reasonable-sounding mitigation, provides real but limited protection: the model's stated confidence is itself a generated output subject to the same fluency-over-accuracy dynamic as everything else it produces, and there is no guarantee its expressed uncertainty tracks its actual error rate on that specific calculation. Confidence flagging is worth doing, since it costs nothing and occasionally surfaces a genuine hedge, but it should be treated as a weak signal to be verified, not a reliable early-warning system that can substitute for the source-verification protocol described elsewhere in this article.
A Worked Case: The Plausible Wrong Number
A mid-sized firm used an AI tool to draft a quarterly board narrative summarizing revenue performance across three business lines. The tool was given access to the underlying revenue figures and asked to calculate and narrate each line's contribution margin. Two of the three margin calculations were correct. The third, for the smallest and least frequently discussed business line, was overstated by several percentage points, the result of the model incorrectly allocating a shared overhead figure across the three lines during its internal calculation, an error invisible in the final prose, which read with identical fluency and confidence across all three figures.
The error was caught not because a reviewer noticed anything unusual in the writing, there was nothing stylistically to notice, but because the firm's protocol required every margin figure in board materials to be independently verified against the accounting system before circulation, a rule adopted specifically because of the kind of collapse the FAITH benchmark documents. Without that specific, mechanical check, the overstated margin would very plausibly have reached the board unchallenged, not due to any reviewer's carelessness, but because the narrative gave them no reason, stylistically or structurally, to look more closely at that particular number than any other.
What This Means For Businesses Without Big Four Resources
The largest firms addressing this problem are building custom verification pipelines, uncertainty estimation models, and dedicated hallucination-detection infrastructure, resources genuinely out of reach for most small and mid-sized Canadian businesses. The good news is that the core protective practice does not require that infrastructure. It requires a specific, enforced discipline: numbers are computed once, in a verified system, and provided to the AI as fixed inputs it narrates but never recalculates, and every narrated figure carries a traceable citation back to that source. This is a process control, not a technology purchase, and it is available to any business regardless of its AI budget. The businesses most exposed to this risk are not the ones with the least sophisticated tools; they are the ones using sophisticated tools without this specific, disciplined separation between what the AI is allowed to compute and what it is only allowed to describe.
The Numbers At A Glance
For quick reference: the FAITH benchmark found top models at 95.6% accuracy on simple financial lookups, collapsing to near-zero on multivariate calculations. Perturbed financial statement testing found accuracy dropping to approximately random chance once surface-level pattern similarity to training data was removed. The EU AI Act's high-risk system obligations reach a key compliance milestone on August 2, 2026. Canada's Artificial Intelligence and Data Act died on the order paper January 6, 2025 and has not been reintroduced. CIBC's own 2026 research paper on hallucination management treats the problem as a continuous operational discipline, not a solved one.
The Limits Of This Analysis
Several caveats matter. Benchmark research in this area, including FAITH and related studies, evolves quickly, and specific accuracy figures cited here reflect the models and conditions tested at the time of publication; newer models may show different, and hopefully improved, performance on the same benchmarks, though the underlying architectural mechanism producing the arithmetic gap described in this article is not obviously resolved by scale alone and has persisted across successive model generations to date. The CIBC research cited addresses hallucination management as an operational framework rather than a definitive solution, and the field broadly describes hallucination mitigation as an ongoing discipline rather than a solved problem. Finally, this article's protocol recommendations are a synthesis of the cited research's implications rather than a peer-reviewed, independently validated set of best practices in their own right, and businesses implementing AI in financial reporting workflows should treat them as a starting framework to adapt, not a compliance checklist.
Frequently Asked Questions
Can better prompting make AI-calculated financial figures reliable?
Does retrieval-augmented generation (RAG) solve the hallucination problem?
Why is AI-generated narrative more dangerous than an AI-generated spreadsheet error?
What's the single most effective practical fix?
Does Canada regulate AI use in financial reporting?
References
- FAITH: A Framework for Assessing Intrinsic Tabular Hallucinations in Finance. (2025). arXiv preprint, arXiv:2508.05201.
- FinVerBench: Benchmark Validity and Calibration in Large Language Model Financial Statement Verification. (2026). arXiv preprint, arXiv:2605.29586.
- Kang, H., & Liu, X.-Y. (2023). Deficiency of Large Language Models in Finance: An Empirical Examination of Hallucination. I Can't Believe It's Not Better Workshop: Failure Modes in the Age of Foundation Models, NeurIPS.
- Agand, P. (2026). Neuro-Symbolic Approaches to Deterministic Fact Grounding in Financial Language Models. Preprint cited in FinVerBench (2026), arXiv:2605.29586.
- Lopez-Lira, A., et al. (2025). Robustness of Large Language Models to Perturbed Financial Statements: Evidence of Memorization over Reasoning. Cited in FinVerBench (2026), arXiv:2605.29586.
- Baltic Assist. (2026). AI Hallucinations In Financial Reporting: Why 'Human-in-the-Loop' Nearshoring Is The Safe Bet. balticassist.com/.../ai-hallucinations-in-financial-reporting
- Pesaranghader, A., & Li, E. (2026). Hallucination Detection and Mitigation in Large Language Models. CIBC. arXiv preprint, arXiv:2601.09929.
- European Union. (2024). Regulation (EU) 2024/1689 (Artificial Intelligence Act). High-risk system obligations effective milestone, August 2, 2026.
This article discusses current AI reliability research and is provided for general informational purposes. It is not a product recommendation, compliance advice, or a guarantee of any AI tool's performance. Confirm the specific capabilities, limitations, and appropriate use of any AI financial tool with its provider and a qualified accountant before relying on it for material financial reporting.