“Training a model” and “model weights” are two of the most common phrases in AI coverage — and two of the least explained. Here’s what’s actually happening when a company says it “trained” a new model, and what those weights everyone talks about actually are.
What Are Model Weights?
Weights are the numerical values inside a neural network that determine how much influence one piece of information has on another as data flows through the model. Picture a neural network as a web of connections between artificial “neurons” — each connection has a weight attached to it, essentially a number that says how strongly that connection should matter.
Weights are the majority of what people mean when they refer to a model’s “parameters” (see our article on AI parameters for more). A model with 70 billion parameters has roughly 70 billion of these adjustable numbers distributed across its network.
What Does “Training” Actually Mean?
Before training, a model’s weights are typically set to small random values — the model knows nothing. Training is the process of gradually adjusting those weights so the model’s outputs get closer and closer to correct, useful answers.
Here’s the basic cycle, repeated an enormous number of times:
- The model makes a prediction (for a language model, often predicting the next word in a sentence).
- That prediction is compared to the actual correct answer, producing an “error” score.
- A technique called backpropagation calculates exactly how much each individual weight contributed to that error.
- Every weight gets nudged slightly in the direction that would have reduced the error.
Repeated across billions of examples, this process gradually shapes the weights into a configuration that produces genuinely useful, coherent outputs — even though no human ever manually set a single one of those numbers.
Why Training Is So Expensive
Adjusting billions of weights across trillions of words of training data requires enormous amounts of computing power, typically specialized chips called GPUs or TPUs running for weeks or months. This is the primary reason training a large foundation model can cost tens or hundreds of millions of dollars — it’s almost entirely a function of the sheer scale of this weight-adjustment process.
Why This Matters for Understanding AI
Once training is complete, the weights are essentially frozen, encoding everything the model “knows” in that fixed set of numbers. This is also why a model has a knowledge cutoff: it only knows what was in its training data up to the point training stopped, since the weights aren’t being updated in real time as the model is used.
The Bottom Line
Model weights are the adjustable numbers that encode everything a neural network has learned, and training is the long, computationally expensive process of tuning those numbers until the model’s outputs become useful. It’s a mechanical, iterative process of trial, error, and adjustment at a massive scale — not fundamentally different in kind from how a person improves with practice, just vastly faster and more mathematical.