When AI Gets the Numbers Right but the Meaning Wrong: The Risks of AI in Accounting and Auditing
ChatGPT with Stephen D Green, October 2026
Artificial intelligence is increasingly being used to process invoices, check tax calculations, reconcile payments and support financial audits. These systems can save time and identify errors, but an important weakness remains: an AI can perform a calculation correctly while misunderstanding the financial information on which that calculation depends.
Consider a simple example.
An invoice issued in Abeesee in June 2024 shows a taxable amount in the currency code ABC, with the tax payable displayed in US dollars (USD). At the time the invoice was issued, ABC represented Abeesee's national currency. However, when the AI model was trained, it had encountered the same code associated with a different currency.
The AI extracts the figures, applies the stated tax rate and calculates the expected total. Its arithmetic is flawless. Yet it interprets ABC using the wrong historical meaning, potentially applies an inappropriate exchange rate, and rejects an invoice whose tax amount is correct.
The problem is not necessarily the mathematics. It is the assumption behind the mathematics.
The hidden weakness: knowledge is not automatically corrected
Large language models learn from training data and statistical patterns. They do not ordinarily maintain a fully verified, continuously updated ledger of facts. When new information contradicts what a model previously learned, that earlier information does not automatically disappear.
A model may therefore produce a plausible answer based on an outdated currency definition, a superseded tax regulation or an obsolete accounting convention. Even when it recognises that a document is unusual, it may fill gaps with an assumption rather than establish the correct interpretation.
This creates a particular risk in finance, where seemingly minor details—currency codes, invoice dates, jurisdictions, tax classifications and exchange-rate conventions—can determine whether a payment is correct.
Why this matters for business payments
In a payment workflow, an AI's approval may be treated as evidence that an invoice has been checked. But if the underlying interpretation is wrong, the system can give a false sense of security.
Potential consequences include:
- Incorrect tax calculations and tax reporting.
- Overpayments or underpayments caused by inappropriate currency conversions.
- Inaccurate accounting entries and foreign-exchange adjustments.
- Reconciliation failures between invoices, bank payments and general-ledger records.
- Audit findings that go undetected because the checking process relies on the same incorrect assumption.
The danger is not limited to unusual currencies. Tax rules change, companies merge, suppliers change their legal identities, and accounting standards evolve. A system that treats yesterday's knowledge as today's truth can make errors at scale.
Correcting the answer is not the same as correcting the system
Suppose an accountant tells the AI that its interpretation of ABC is wrong. The model may correct its answer in that conversation, but this does not necessarily update its underlying trained knowledge, its external reference database or the rules used in subsequent invoice checks.
Unless the correction reaches the appropriate source of information, the same error may recur when another invoice is processed.
This distinction is crucial for auditing. A correction that exists only in a conversation is not a reliable control. Businesses need to know whether the authoritative record was updated, whether the revised information was used in later transactions, and whether earlier transactions require review.
A safer approach: verify the evidence, not just the arithmetic
A robust AI-assisted accounting process should establish the meaning of financial codes at the invoice date, consult authoritative currency and tax references, apply the relevant accounting rules, and independently recalculate the figures using deterministic software.
It should also flag uncertainty rather than silently inventing missing information. Where appropriate, it should retain an audit trail recording the source data, exchange rate, calculation method, assumptions, correction history and approval decision.
Importantly, the original information should not always be deleted. A currency code may have had one valid meaning in an earlier period and another later. The system should preserve historical accuracy while ensuring that the correct interpretation is applied to each transaction.
Human review remains important for exceptions, ambiguous documents and material financial decisions. AI can help identify and investigate discrepancies, but its confidence should never be mistaken for proof.
Conclusion
AI can be an extremely useful tool in business payments, accounting and auditing, but reliable financial control requires more than a convincing answer or a correct calculation.
The central question is not simply whether the AI can calculate the tax. It is whether the AI has established that it is calculating the right tax, using the right currency, the right rules and the right information for the relevant date.
In finance, an error that looks mathematically precise can still be fundamentally wrong. The objective should therefore be to build AI systems that are evidence-based, time-aware, transparent about uncertainty and independently verifiable—not systems that merely sound certain.
