Why AI is bad at simple maths
A player handed an AI a long multiplication problem. It returned a polished explanation, a confident answer, and one missing carry. This is an unusual failure mode only if you assume the eloquent scholar and the pocket calculator are the same character class. They are not.
Why does AI get simple maths wrong?
A language model can get simple maths wrong because its core task is to predict the next piece of text, not to execute a guaranteed arithmetic procedure. It may have learned strong mathematical patterns and can reason through difficult problems, but an unaided answer is still generated token by token. One incorrect digit can look just as fluent as a correct one.
That distinction matters more than the difficulty label. A model may recognise the structure of a calculus proof it has seen in many forms, yet make an error in a long, unfamiliar multiplication where every carry must be exact. Modern reasoning models are much better at maths than early chatbots, and some products quietly call calculators or run code. The safe rule is not "AI cannot do maths". It is "language output alone is not a calculator receipt".
Fluency can explain the method. Only execution and checking make the number dependable.
A language model learns patterns, not a calculator keypad
During generation, a language model estimates which token should come next from the prompt and the text already produced. Tokens are the chunks described in my field note on what AI tokens are. Training exposes the model to equations, worked solutions, code, tables, and explanations, so it can learn remarkably useful patterns connecting a problem to a likely method and answer.
But learning examples of addition is not the same mechanism as a calculator implementing addition. A calculator follows a fixed algorithm over an exact representation and returns the result. A language model produces a plausible sequence. It can imitate the algorithm, write code that performs it, or call a tool that performs it, but those are different routes with different reliability.
This is why the model's prose can be more convincing than its total. The explanation and the digits are emitted by the same generator. There is no universal internal alarm that rings because the final column failed to balance. A wrong result may simply remain a very probable-looking continuation.
Numbers are chopped into tokens
Before the model handles a number, a tokenizer divides the text into tokens. Depending on the tokenizer and the input, a string of digits may be split one digit at a time or in multi-digit chunks. The model is therefore not necessarily looking at a number in the neat place-value columns a person would draw on paper.
A 2024 study of number tokenisation and arithmetic found that tokenisation choices materially affect arithmetic performance and error patterns. Direction and grouping matter: representing the digits in a way that matches the order of carrying can make the learned task easier. This does not explain every maths failure, but it shows that the text encoding is part of the problem—not an invisible neutral wrapper.
The same boundary issue helps explain why a model may miscount letters in a word. It often processes token chunks rather than scanning a row of individual characters as a tiny human with a pencil. Asking it to write code that iterates over the characters changes the task from predicting a count to executing one.
Each step can be plausible while the chain is wrong
Arithmetic and word problems are unforgiving. If a solution has eight dependent steps and each is merely very likely to be right, one slip can poison everything after it. Natural language is tolerant: swap a synonym and the sentence still works. Arithmetic has one valid product.
Researchers behind GSM-Symbolic tested variations of grade-school word problems and found that model performance could change when numerical values changed or irrelevant clauses were added. The result is a warning about robustness: success on a familiar-looking benchmark does not guarantee that the model has a stable procedure for every altered version.
There is also a parsing stage before calculation. The model must decide what the question means, which facts matter, what operations connect them, and which units belong in the answer. A calculator cannot rescue a wrongly understood question. Good systems separate these jobs: let the model interpret and plan, let deterministic software calculate, then check that the result answers the original request.
Why asking it to "think step by step" only partly helps
Breaking a problem into explicit steps can help because it gives the model more room to organise the task and exposes mistakes a reader can inspect. It is useful for understanding the chosen method. It does not turn probabilistic text generation into verified computation.
A chain of reasoning can contain a copied value, sign error, invented assumption, or incorrect intermediate result. Longer working may even create more places to slip. Treat a written derivation as evidence to inspect, not proof that an unseen mechanism checked every line.
This is related to, but distinct from, why AI makes things up. In both cases the model can produce a well-shaped answer without an external truth test. For maths, we often have an unusually good truth test: execute the calculation.
The reliable fix is to give the AI tools
The best division of labour is simple. Use the language model for the part expressed in language; use a calculator, spreadsheet, symbolic engine, or code runtime for exact operations. The model can translate "apply 17.5% tax after the discount" into steps, but software should carry out those steps.
The Program-Aided Language Models research made this separation explicit: the model decomposes a natural-language problem into a program, while a Python interpreter executes the solution. The important idea has aged well even as models have improved. Reason about the task with the flexible component; compute with the deterministic one.
If you are using a chat product, ask whether it can use a calculator, data-analysis, or code tool, then request that it use the tool and show the inputs. If you are building a workflow at Wistkey or elsewhere, expose a narrowly defined calculation function and validate its arguments before execution. OpenAI's current function-calling guidance describes this pattern: the model requests a function, your application runs it, and the result returns to the model.
How to use AI safely for maths
Match the safeguard to the stakes:
- For a quick low-stakes sum, use the calculator on your device. Asking a chatbot adds an unnecessary translation layer.
- For a word problem, ask the AI to identify the variables, assumptions, units, and formula. Then calculate with a tool.
- For a spreadsheet, let the AI propose the formula, but test it on rows whose answers you already know and inspect the cell references.
- For statistics, finance, engineering, or science, verify the method as well as the arithmetic. Correct execution of the wrong model is still wrong.
- For repeated production work, move the calculation into deterministic code, add range and unit checks, and write tests around edge cases.
- For teaching, use the explanation to explore an approach, then compare each step with a textbook, teacher, or trusted solution. Do not grade the confidence of the prose.
You can also ask for a rough magnitude before the exact answer. If 63 items cost about £20 each, a claimed total of £126 is visibly in the wrong region. Estimation catches many failures cheaply. Independent recalculation catches more.
The practical answer
AI can be excellent at choosing a mathematical approach, explaining concepts, translating a question into equations, and writing code to solve it. It becomes dependable when the exact computation is executed and checked by a suitable tool. Do not confuse a model's ability to discuss maths with a guarantee that every generated digit is correct.
I have updated the party configuration. The language model may read the quest, interpret the ancient inscription, and explain why the bridge needs 63 planks. The calculator will count the planks. Nobody receives extra experience for falling through plank 62 with an eloquent justification.
