RAG and File Search

today-i-learned artificial-intelligence

This document will be updated over time.

RAG

  • Processes documents
  • Stores as chunks in a database
  • Answers generated based on what’s the semantically nearest chunk to the user prompt
  • Looks through documents during runtime
  • Attempts to look for what we exactly asked for

Comparison

It looks to me that RAG is suitable when:

  • We need semantic matching
  • There are more than a few hundred documents’ worth of data to look through in a single session
  • We have the capacity to set up indexing of documents

Whilst file search is suitable when:

  • Keyword matching is enough
  • A small scope of documents to look through e.g. less than a few hundred documents
  • We prefer less maintenance and don’t require optimal performance

I’m mentioning a few hundred documents as that is what my search results are saying – I could be wrong, though.