Train a language model in your browser
A small GPT — tokenizer, self-attention, backpropagation and AdamW — written from scratch in TypeScript and compiled to the plain JavaScript your browser runs. No framework, no bundler, no runtime dependencies, no server, no API key. Paste some text, press one button, watch the loss fall, then test the model you just made.
- Nothing you type leaves this page. It trains in a Web Worker on your own machine.
- Seconds, not hours. About thirteen thousand to three hundred and thirty thousand parameters, depending on how long your text is, trained on your own machine in about twenty seconds to five minutes.
- Read all of it. The compiled model, and the TypeScript it is built from.
1 Give it something to learn
A small model learns whatever regularity is in front of it. Small and highly structured works best: a list of names, a list of products, a script with repeated turns.
2 Choose the size of the model
Bigger is slower — and worth it only if it is allowed to finish. Every preset here is trained long enough to produce readable text — not just the right vocabulary in the wrong order — which is why none of them is instant. The estimate next to each is measured on a desktop CPU; yours may differ, and the page reports the rate it is really achieving once it starts.
3 Create the model
This is real training: forward pass, backward pass, and an optimiser step, over and over, on your machine. You can stop it at any point and test what it has learned so far — but stopping early is the one thing that produces nonsense: until the loss is low the model has learned which words (or letters) are common and not yet which follow which, so it emits the right pieces in the wrong order. Press Sample from it when it stops and the page will tell you whether what came out is readable.
- loss
- —
- step
- —
- throughput
- —
- tokens seen
- —
- time left
- —
4 Test the model you made
Train a model first — then this panel lets you sample from the one you just created.
How to tell whether it worked
Working: real words in the right order, and the shape of your own document — its line breaks, its names, its formatting. A list of names comes back as new names that could have been in the list. Prose comes back as sentences that read.
Not there yet: real letters in nonsense order, or the same line over and over, or one wall of text with no line breaks. That means it stopped before it had trained enough — run a bigger preset, or the same preset on a shorter text.
The verdict above is measured, not an opinion: it counts how much of what you are looking at also occurs in the text you gave it, and compares that with the bar its own test suite holds the build to.
Vocabulary the model actually learned from your text:
How it works
The whole thing, in the order the data moves through it.
- Tokeniser — the level follows your text. The vocabulary is built from what you paste. A document of prose is split into words, because words repeat and a word is the unit a reader reads. A list — names, products, one item per line — is split into characters, because there every “word” occurs once, so a word vocabulary would just be a copy of your list and the model could only recite it; at character level it can compose new entries from real letters instead. You are never asked which — the shape of the text decides, and the model panel shows which one it picked.
- Embeddings. Each token becomes a vector, and a second vector is added for its position, so the model knows the difference between ma and am.
- Transformer blocks. Each block is: layer norm → causal multi-head self-attention → residual add → layer norm → feed-forward with GELU → residual add. Causal means a position can only see the positions before it, which is what makes it a predictor rather than a copier.
- Head and loss. A final layer norm, then a linear layer to the vocabulary, then softmax and cross-entropy against the next character. That number — the loss — is what falls as it learns.
- Backpropagation. Every gradient is derived by hand and written out in the file: through the attention, through both layer norms, through GELU, through the embeddings. The test suite checks them against numerical gradients, so a wrong derivative fails the build rather than silently training a broken model.
- AdamW. Adaptive steps with decoupled weight decay, plus a warm-up of the initialisation scaled by the depth — the same choices the big models make, at ten-thousandth the size.
- Sampling. Temperature rescales the logits and top-k truncates the distribution before drawing, which is the difference between a model that repeats itself and one that improvises.
What this is, and what it is not
It is
- A real transformer, with real attention and real backpropagation — not a call to somebody else's API.
- Real training, running in front of you: the loss curve is measured, not drawn.
- Deterministic. The same text, size and seed produce the same model every time.
- Yours to keep: the weights download as plain JSON.
It is not
- A large language model. It is about thirteen thousand to three hundred and thirty thousand parameters; GPT-2 small is 124 million, and a frontier model is larger again by orders of magnitude.
- Trained on the world. It only knows the text you gave it — outside that text it reproduces the shapes it learned, which is the honest failure mode of a tiny model.
- Comprehending anything. Trained to the end it reproduces your text very closely, and that is memorisation — the right outcome for a demo this size. It does not understand what it is writing, and it cannot answer a question about the text.
- A substitute for a GPU. One CPU core in a browser is the whole budget, which is exactly why the demo is built this way: the mechanism is the point.
Privacy
The text you paste is trained on in a Web Worker on your own machine and is never uploaded — there is no request in the model code that could carry it, and a test asserts that. The model exists only in this tab and disappears when you close it. That is true whatever you answer about cookies.
One thing does leave the page, and none of it is yours: if the page throws a JavaScript error, the error is reported to this site's own address so it can be fixed — the browser, the page, the message and the line it came from. Never the text you type or train on, and not even your own address: the report is relayed by our own server, so what reaches the error tracker comes from that server and not from you.
Cookies
There are no accounts here and nothing is stored about you. That leaves one cookie, and it is optional: Google Analytics, which records anonymous page, click and scroll events — never anything you type, and never the text you train on. Google's script is not in this page at all. It is added only if you allow it, so a visit that rejects makes no request to Google and receives no cookie.
Your answer is remembered in your own browser's storage, not in a cookie — setting a cookie to record a cookie decision would be its own small joke. You can change it whenever you like from Cookie settings in the footer, which opens the same panel that Settings does while the question is up.