nanoGPT Arithmetic Explorer: A small MATLAB GPT that groks integer addition
A small MATLAB GPT that groks integer additionDuncan Carlsmith, Department of Physics, University of Wisconsin-Madison
Introduction
My prior post A miniature GPT language model as a MATLAB Live Script provides a small character-level GPT in MATLAB and trains it on Shakespeare. Language modeling is intractable in the sense that there is no exact answer to grade against; the model is judged by whether its output reads plausibly. This companion Live Script applies the same architecture to a problem with a clear success metric. The goal of training is for the model to discover/get/“grok” an exact algorithm for integer addition in any base (e.g., binary, decimal, and hexadecimal). With this script, you can watch a “grok,” the model suddenly “getting” addition, happen in real time and in detail- very fun if you are like me, hardly an expert!
Addition is easy except for the carry. The digit in a column is the column sum modulo the base. The carry couples columns: a carry out of one column feeds the next, and consecutive carries chain groups of columns together. The script grades the model on held-out problems sorted by the length of the longest carry chain, so the per-chain accuracy curves show exactly where and how the model succeeds. One observes the model first grok single carries, then singles and doubles, and so on, up to the maximum for the range of digits presented during training. In representing a finite range of numbers, the distribution of carry chain lengths depends on the base, as does the number of essential tokens. Hence, although the essence of addition is base-independent, the learning curves are different for different bases.
As is well known, the grok duration can be very short or gradual, depending on exactly how the addition problem is posed when presented to the model and if a random or structured learning curriculum is used. The script demonstrates that since a random number generator based on a seed is used to generate training samples, the time and even existence of a grok in a prescribed length set of training steps fluctuates, sometimes wildly, but for the same seed, and training is reproducible deterministically to machine precision. Comparative studies of the efficacy of different model sizes, problem presentation formats, and training curricula require statistical analysis and are not attempted by this script.
The Live Script is organized as five experiments. The first trains naively on random problems and can show a vexing carry stall - a partial grok. The next two use a difficulty curriculum that orders training from short carry chains to long ones, and a scratchpad format that writes each column's digit and carry explicitly so the carry is a token in context rather than something inferred. The fourth combines both. The fifth trains one model on a mix of bases, each problem labeled with its base, and tests a base never seen in training; the base-independent column sum transfers to the new base, but the base-specific carry threshold does not. Each experiment displays a precomputed figure by default. Each also has a "Try this" switch that retrains it live with your own base, digit count, model size, and seed. A set of challenges asks the reader to extend the work, for example, to test length generalization or study subtraction together with addition.
The engine that does the training and evaluation is a set of MATLAB functions shipped with the script and usable on their own from the Command Window, independent of the Live Script. An accompanying guide documents them for readers who want to understand the details or run their own experiments. The script and engine were built with Claude (Anthropic) working with MATLAB R2026a on my own MacBook with an M1 chip through an ngrok command server, the agentic context described in my prior posts. No GPU or API is used.
References
Duncan Carlsmith (2026). nanoGPT Arithmetic Explorer (https://www.mathworks.com/matlabcentral/fileexchange/184054-nanogpt-arithmetic-explorer), MATLAB Central File Exchange. Retrieved June 11, 2026.
Conflict of interest
The author declares he has no financial interest in MathWorks or Anthropic. This article is informational and does not constitute an endorsement by the University of Wisconsin-Madison of any vendor or product. Claude is a trademark of Anthropic. MATLAB is a trademark of MathWorks.
Tags
