A miniature GPT language model as a MATLAB Live Script
A miniature GPT language model MATLAB Live Script
Duncan Carlsmith, Department of Physics, University of Wisconsin-Madison
I have built many MATLAB Live Scripts that take some piece of physics, computation, or machine learning apart so a student can see how it works. The newest one builds, trains, and runs a small GPT language model and was built with AI assistance and may interest readers of this forum.
The model is a MATLAB implementation modeled on nanoGPT, a compact GPT (Generative Pre-trained Transformer) written in Python by Andrej Karpathy and released under the MIT license. This small model is a GPT in each respect: it is generative, producing text one character at a time rather than classifying text; it is pre-trained, trained once on a body of text so that the trained weights can then be reused; and it is a transformer, built from the stack of masked self-attention and feed-forward layers described in more detail in the Live Script Background section. The script trains a character-level model with about 112,000 parameters, two transformer blocks, and one or four attention heads. It is trained on an approximately one-million-character corpus of Shakespeare in roughly twenty minutes on a laptop and, remarkably, generates Shakespeare-flavored text one character at a time. The model parameter count and training corpus pale in comparison to frontier models with estimated parameter counts of order one trillion trained and tens of trillions of tokens. The aim of the Live Script is explanation, not performance.
MATLAB implementation
The training script is vectorized: weights are created as dlarray objects, the loss is computed under dlfeval, and a single call to dlgradient returns the gradient with respect to every weight. The attention of a whole batch of sequences is computed in one pagemtimes call rather than a loop. The autodiff and the batched array arithmetic are what make training on a laptop practical.
A note on the development
The script, the model class, and the trainer were built with Claude (Anthropic), starting with the Python and working with MATLAB R2026a on my own machine through an ngrok command server.
Try this and challenge options
The script ships with the Shakespeare text and one pre-trained model. It also includes an optional section that downloads three public-domain novels from Project Gutenberg — Conan Doyle, Wells, Austen — strips the license boilerplate, and aggregates them into a single corpus, so a student can train a model that writes in a different voice. A model trained on novels does not sound like one trained on Shakespeare, and that difference is the most direct lesson the script offers about what these models actually learn. Several challenges are offered to expand the model and the demonstration.
The submission is on the MATLAB Central File Exchange: Duncan Carlsmith (2026). nanoGPT Explorer (https://www.mathworks.com/matlabcentral/fileexchange/183953-nanogpt-explorer), MATLAB Central File Exchange. Retrieved May 24, 2026.
Conflict of interest
The author declares he has no financial interest in MathWorks or Anthropic. This article is informational and does not constitute an endorsement by the University of Wisconsin-Madison of any vendor or product. Claude is a trademark of Anthropic. MATLAB is a trademark of MathWorks.
1 Comment
Time Descending
For some reason, the link to the File Exchange entry isnt' actually a link. Here it is https://www.mathworks.com/matlabcentral/fileexchange/183953-nanogpt-explorer
Sign in to participate
Tags
