The Librarian, the Card Catalog, and RAG

I was sitting in an FDE workshop today when we were given what sounded like a pretty simple exercise: explain RAG as if you were explaining it to a customer.

Retrieval-Augmented Generation. We work with it. We design solutions around it. Surely explaining what it actually does should be the easy part.

It was surprisingly hard.

Around the room, people tried different approaches. Someone compared it to a tourist information kiosk. Someone talked about addresses. Others went straight to search indexes, vector search, chunking, and, inevitably, math.

None of those explanations were wrong. But the more we talked, the more I realized how difficult it is to explain RAG in plain language without immediately introducing five more things you also have to explain.

At some point, a library came up in the conversation. And that one stuck with me.

Not today’s library, where I assume you can just search for everything on a computer. I’m talking about the library I learned to use as a kid, back when “search” involved a giant wooden cabinet full of tiny drawers.

Yes, children. The search engine was furniture. Please enjoy your youth and allow the rest of us to continue.

Learning to Find the Book

I remember learning how to find things in the library. It wasn’t enough to know what book I wanted. First, I had to figure out where the library had put it.

The books weren’t randomly piled onto shelves. They had been organized. The Dewey Decimal System grouped books by subject and gave them a place in the collection. The card catalog helped me figure out what the library had and where I could find it.

I’d look through those little cards, find the number I needed, and head into the stacks. Then I’d walk along the shelves looking at the numbers until I found the book.

But finding the book still didn’t necessarily mean I’d found the information. Maybe I needed one chapter. Maybe I needed three pages. So I’d use the table of contents, headings, chapters, or index to narrow things down again.

There were systems for finding the right book, and then there were systems for finding the right information inside the book.

That’s when RAG started to make a lot more sense to me.

What If I Had a Librarian Do All of That for Me?

Imagine I walk into our library and instead of going to the card catalog myself, I walk up to a librarian and ask:

“Why are the leaves on my tomato plants turning yellow?”

I don’t know which book has the answer. I don’t know the author. I definitely don’t know the Dewey Decimal number.

I just know what I’m trying to figure out.

Our librarian takes that question and goes looking. Maybe there are books about vegetable gardening, plant diseases, watering, soil nutrients, or gardening in extreme heat. The exact words I used don’t necessarily have to match the title of a book.

The librarian figures out which parts of the collection are likely to be relevant, finds the useful information, and brings it back.

They also don’t return carrying six 400-page gardening books and drop them on the desk with, “Here you go. It’s in there somewhere.”

At least I hope not. That’s a terrible librarian.

Instead, they might come back with a chapter from one book and a few pages from another. They found the pieces of information that were most relevant to my question.

Then they use that information to give me an answer.

That’s RAG.

R. A. G.

Once I had the library in my head, even the name started to feel less complicated.

The librarian searches the collection and finds the information relevant to my question. Retrieval.

They bring that information back and add it to the context of our conversation. Augmentation.

They use my question and the information they found to formulate an answer. Generation.

Retrieve. Augment. Generate.

Our actual AI isn’t running around carrying books, of course, and what’s happening behind the scenes is much more sophisticated than my childhood card catalog.

Modern retrieval can search based on meaning, not just whether I happened to use the exact right words. If I ask why my plants keep dying when it’s 115 degrees outside, useful information might be stored under “heat stress in desert plants.” Semantic search helps connect those ideas even though I didn’t phrase my question that way.

Embeddings and some very clever math help make that possible.

I’m going to leave the math to people who enjoy that sort of thing.

For our purposes, the important part is that I don’t have to know what the information is called or where it lives. I can tell the librarian what I’m trying to understand, and they can look for information that means something similar.

What If There Isn’t a Book?

There’s one more reason I really like thinking about RAG this way.

What happens if our librarian searches the collection and can’t find anything that answers my question?

A good librarian tells me that.

They don’t disappear into the back room, write a brand-new book, sneak it onto the shelf, and then confidently return to tell me what they “found.”

As ridiculous as that mental image is, it’s an important part of why we care about grounding AI responses in trusted information. Retrieval gives the model relevant information to use when generating its answer. It doesn’t guarantee that AI can never be wrong, but it gives us a much better foundation for an answer than simply hoping the model already knows what we want it to know.

And that’s where I started thinking less about our librarian and more about the library.

What Kind of Library Did We Give Them?

We’ve spent a lot of time making our librarian smarter. But what happens when we send them into the stacks?

What happens if there are three versions of the same policy sitting on the shelf, and nobody has identified which one is current?

What happens when two books contradict each other? What if a 400-page manual has no chapters, headings, or meaningful structure? What if a book is sitting in the wrong section? What if important information isn’t in the library at all?

What happens when our librarian finds exactly the right answer, but it’s in a book the person asking the question isn’t supposed to be able to read?

And who is making sure the library stays organized as we keep adding more and more information to it?

Those aren’t really AI questions.

They’re architecture questions.

And they’re exactly the kinds of questions I’ve been exploring recently while working with Salesforce Knowledge and AI.

Because if we’re going to ask AI to be our librarian, we should probably spend some time thinking about the library we’re giving it.

In a way, this brings me right back to what I’ve been writing about in my Back to Basics series. We can talk about AI as something new, but so much of making it work well comes back to the same architecture fundamentals we’ve always cared about. Structure your information well. Make intentional decisions about security and access. Create governance. Build a foundation you can trust. The better we architect the environment underneath AI, the better chance we have of building something useful on top of it.

I’ve got a lot of that knowledge rattling around in my brain right now. I’ll work on getting it out of there and onto the blog soon.