Is this a real language model?

Yes. It is a transformer with token and position embeddings, causal multi-head self-attention, a feed-forward block, layer norms and residual connections, trained by backpropagation with AdamW. It is small — thousands to hundreds of thousands of parameters — and it trains in your browser on one CPU core in seconds.

Does the text I paste get uploaded?

No. The text is trained on inside a Web Worker on your own machine. There is no server-side model, no API key, and no request carrying your text. The page carries Google Analytics, which records anonymous page, click and scroll events only.

Why is the model so small?

Because one browser tab on one CPU core is the budget. A frontier model is trained on thousands of accelerators for months. This demonstrates the mechanism — tokenisation, attention, backpropagation, optimisation, sampling — and not the scale, and the page says so rather than pretending otherwise.

Can I keep the model I trained?

Yes. The weights download as plain JSON, with the vocabulary and the exact configuration, and a training report with the loss curve can be downloaded beside it.

Train a language model in your browser

A small GPT — tokenizer, self-attention, backpropagation and AdamW — written from scratch in plain JavaScript. No framework, no build step, no server, no API key. Paste some text, press one button, watch the loss fall, then test the model you just made.

Nothing you type leaves this page. It trains in a Web Worker on your own machine.
Seconds, not hours. About twelve thousand parameters in about fifteen seconds.
The whole model is one readable file. llm.js — read it.

1 Give it something to learn

A small model learns whatever regularity is in front of it. Small and highly structured works best: a list of names, a list of products, a script with repeated turns.

Training text

2 Choose the size of the model

Bigger is better here too — and slower. The estimate next to each is measured on a desktop CPU; yours may differ, and the page will tell you the rate it is really achieving once it starts.

3 Create the model

This is real training: forward pass, backward pass, and an optimiser step, over and over, on your machine. You can stop it at any point and test what it has learned so far.

loss: —
step: —
throughput: —
tokens seen: —
time left: —

4 Test the model you made

Train a model first — then this panel lets you sample from the one you just created.

Start it off (optional)

Temperature 0.80 Top-k 0 Length 200

Vocabulary the model actually learned from your text:

How it works

The whole thing, in the order the data moves through it.

Tokeniser — character level. The vocabulary is built from your text: every distinct character becomes a token. No vocabulary file is downloaded, because the text makes its own.
Embeddings. Each token becomes a vector, and a second vector is added for its position, so the model knows the difference between ma and am.
Transformer blocks. Each block is: layer norm → causal multi-head self-attention → residual add → layer norm → feed-forward with GELU → residual add. Causal means a position can only see the positions before it, which is what makes it a predictor rather than a copier.
Head and loss. A final layer norm, then a linear layer to the vocabulary, then softmax and cross-entropy against the next character. That number — the loss — is what falls as it learns.
Backpropagation. Every gradient is derived by hand and written out in the file: through the attention, through both layer norms, through GELU, through the embeddings. The test suite checks them against numerical gradients, so a wrong derivative fails the build rather than silently training a broken model.
AdamW. Adaptive steps with decoupled weight decay, plus a warm-up of the initialisation scaled by the depth — the same choices the big models make, at ten-thousandth the size.
Sampling. Temperature rescales the logits and top-k truncates the distribution before drawing, which is the difference between a model that repeats itself and one that improvises.

What this is, and what it is not

It is

A real transformer, with real attention and real backpropagation — not a call to somebody else's API.
Real training, running in front of you: the loss curve is measured, not drawn.
Deterministic. The same text, size and seed produce the same model every time.
Yours to keep: the weights download as plain JSON.

It is not

A large language model. It is thousands to hundreds of thousands of parameters; GPT-2 small is 124 million, and a frontier model is larger again by orders of magnitude.
Trained on the world. It only knows the text you gave it — outside that text it produces plausible-looking noise, which is the honest failure mode of a tiny model.
A substitute for a GPU. One CPU core in a browser is the whole budget, which is exactly why the demo is built this way: the mechanism is the point.

Privacy

The text you paste is trained on in a Web Worker on your own machine and is never uploaded — there is no request in the model code that could carry it, and a test asserts that. The model exists only in this tab and disappears when you close it.

The page carries Google Analytics, which records anonymous page, click and scroll events — never anything you type. Stop counting this browser · resume.