Lai Learns Neural Networks

Lesson 11·Marathon training·4 min read

Of Laps Run, and Laps Run Again

The training loop and epochs: why networks learn by repetition

“He did not notice the morning he became a runner. There was only the habit of the same road, taken again and again, until one day the distance that had once frightened him passed beneath his feet like a familiar thought.”

Invented, in the spirit of Stefan Zweig

Welcome back

Over the last few lessons, we’ve gathered all the pieces of learning. The network makes a guess (the forward pass). We measure how wrong it was (the loss). We trace the blame backward (backpropagation, powered by the chain rule). Then we nudge every weight a little downhill (gradient descent, with a sensible learning rate).

Today we put those pieces into a rhythm. And there’s no better teacher of rhythm than a marathon runner.

Nobody runs 42 kilometres on day one

Imagine a friend who has decided to run their first marathon. On the first morning, they jog for twenty minutes and come home puffing. They don’t run a marathon that day. They don’t expect to.

Instead, they go out again the next day. And the next. They run familiar routes, over and over. Each run changes their body only a tiny bit: a slightly stronger heart, slightly steadier legs. No single run makes them a marathoner. But months of the same routine, repeated with patience, quietly does.

A neural network learns in exactly this way.

The training loop

At the heart of every network’s education is a simple routine called the training loop. It goes like this:

  1. Show the network an example.
  2. Guess: the network makes a prediction.
  3. Measure: the loss tells us how far off the guess was.
  4. Trace: backpropagation works out each weight’s share of the blame.
  5. Adjust: each weight takes a small step to reduce the loss.

Then it goes back to step one with the next example, and round again.

Each trip around the loop is like a single training run. It changes the network only slightly. Remember the learning rate, our gentle hand on the shower tap? It keeps each adjustment small on purpose, so the network doesn’t lurch from one extreme to another.

Epochs: one lap of the whole course

Suppose we have ten thousand examples to learn from. When the network has worked through all ten thousand once, we say it has completed one epoch. It’s like the runner finishing one full lap of their usual route.

One epoch is rarely enough. After a single lap, the network has only begun to notice the patterns. So we send it round again, over the very same examples, for a second epoch, then a third, then perhaps fifty or a hundred more.

You might wonder: why would seeing the same examples again help? Because each time, the network is a little different. It comes back with slightly better weights, notices things it missed before, and corrects mistakes it made last time. Just as the runner’s familiar road feels easier every week, the same examples teach a little more every lap.

How do we know it’s working?

Runners watch their times drop. Networks watch their loss drop. If you draw the loss after each epoch, you’ll usually see a curve that falls steeply at first, then more gently, then levels off. That flattening is the network saying, “I’ve learned most of what these examples can teach me.”

What to carry with you

In one line

A network learns by repeating one simple loop (guess, measure, trace, adjust) thousands of times. One full pass through all the examples is called an epoch, and training usually takes many of them.

Next time

We’ll sit at a school desk. A good teacher never lets students see the final exam before the day it’s given. We’ll learn about training, validation and test data, and why keeping some examples hidden is how we know a network has truly learned.