Don't put the knowledge in the model
A model's weights are frozen at training time. It has never seen your Confluence, your contracts, or last Tuesday's incident report. Ask it about your refund policy and it will do the most plausible thing available: write a refund policy that sounds like the ones it has seen. Fluent, confident, well-structured, and not yours.
Fine-tuning is one answer, and it's usually the wrong one. It's slow and expensive, it has to be redone every time a fact changes, and it teaches style far more reliably than it teaches facts. You cannot retrain a model every time someone edits a policy page.
RAG takes a different route entirely. Don't put the knowledge in the model. Put it in the prompt.
The loop
- Retrieve. Take the user's question, search your own documents, pull back the handful of passages most likely to contain the answer.
- Augment. Paste those passages into the prompt: "Using only the context below, answer the question."
- Generate. The model answers — from text sitting directly in front of it.
The model still doesn't know your refund policy. It doesn't need to. It can read.
And because the answer came from a specific retrieved passage, you can show the user that passage. Citation isn't a bolt-on feature here; it falls out of the architecture for free.