Pretraining vs Fine-tuning

People usually meet these as "the big expensive one" and "the small cheap one". That framing is the source of nearly every costly mistake in this area. They aren't two sizes of the same thing. They're two different jobs.

The new hire who has read everything

Imagine hiring someone who has read every book, paper, manual, forum thread and shopping list ever written — and has never held a job in their life.

They know an extraordinary amount. Ask them anything and something relevant comes back.

Now sit them at a desk on Monday morning. They're useless — but in a very specific way. They don't know how we do things here. Ask them a question and they might answer it, or they might carry on writing three more questions underneath yours, because a page of questions is also a perfectly plausible document.

That's a base model: what comes out of pretraining. Enormous capability, no job.

Fine-tuning is their first two weeks. Nobody teaches them new facts in those two weeks. Someone shows them the house style, what a good answer looks like here, when to ask and when to stop, the tone we use with customers. Two weeks cannot compete with a lifetime of reading — and it isn't trying to. It's aiming it.

Pretraining builds the ability. Fine-tuning aims it.

Keep that sentence. Almost every expensive misstep here is somebody expecting fine-tuning to do pretraining's job.