Notes on practice · Essay 05
Weights
On how a model learns, and why it looked like contest prep.
I learned how a model learns by building the smallest one there is: one neuron, the example Andrej Karpathy uses in micrograd. A few inputs, a few weights, one output, one target. I expected it to feel foreign. It felt like contest prep.
The loop
A model learns in four moves. It runs forward and produces an answer. It measures how far that answer missed the target. It traces the miss backward to find which weights caused it. Then it nudges each weight a little and runs again.
Prep is the same loop. You write the program and the athlete runs the week. Check-in tells you how far the week landed from where it needed to be. You work out why. You adjust, and the next week starts.
Nobody gets to the stage in one adjustment. A model doesn't get there in one step either. It gets there in hundreds of small corrections, each one based on what the last one actually did.
One neuron, the example Andrej Karpathy uses in micrograd. Each box is a number, and the lines show how they combine. The goal is an output of 1. Step through it.
A lever that did nothing gets no blame
In the neuron I built, one input was zero. Its weight got no correction at all, because nothing it did could have changed the result. The math refuses to blame a lever that didn't move.
Coaches get this wrong all the time. The client missed, so everything changes at once: food, cardio, steps, training split. Half of those weren't followed that week. You can't learn anything from a lever nobody pulled. You find out what was actually done, and you only adjust what actually ran.
Step size
Every correction has a size. In a model it's called the learning rate. If it's too small, progress crawls. If it's too big, the model overshoots, swings past the target, and can bounce around it without settling.
In prep that's the athlete who drops carbs hard after one bad check-in, goes flat, rebounds, and spends the next three weeks undoing one big reaction. The fix is the same in both places: small, proportional moves, repeated.
One reading is noise
One week's scale weight carries water, sodium, sleep and stress. One batch of training data carries noise too. Neither is the truth on its own. You trust the trend over several rounds and you don't rebuild the program off one morning.
You choose what counts as a miss
A model only gets better at what you measure. If the loss counts the wrong thing, the model gets very good at the wrong thing.
Coach to the scale alone and you get a lighter athlete, not a better one. The judges don't score a number. They score the look under the lights, so the check-in has to measure condition, fullness and balance. Weight is one reading among them.
Weights
Then there's the word. In bodybuilding we spend our lives under weights. In a model, the weights are what it learns, the numbers that decide how heavy each input counts.
That's what training is, in both places. A beginner treats every input as if it weighs the same. Experience teaches you which ones are heavy. The program sets the weights, the work tests them, and the honest measurement afterward decides how much each one counts next time.
I didn't learn this in a computer science program. I learned it across from athletes for years, adjusting one week at a time. It turns out that was the same lesson.