When RAG is useful and when it is not

RAG is useful when the assistant must answer from changing business data, internal documents, user-specific records or domain material that should not be baked into the model prompt.

It is less useful when the answer is generic, the data is tiny, or the workflow needs deterministic business logic more than semantic search.

A production RAG system is not just embeddings. It needs ingestion rules, chunking, permissions, source attribution, logging, evaluation and a clear fallback when retrieval is weak.