Translations:FACTS About Building Retrieval Augmented Generation-based Chatbots/25/en

Metadata Enrichment, Chunking, Query Rephrasal, Query Reranking: We noticed that metadata enrichment, chunking, query rephrasal and query re-ranking stages of RAG pipeline have the most impact on the quality of Chatbot responses. LLM response generation quality is highly dependent on retrieval relevancy. Retrieval relevancy is, in turn, highly dependent on document metadata enrichment, chunking, and query rephrasal. We implemented grid search-based auto-ML capabilities to find the right configurations of chunk token-sizes, experimented with various prompt variations, and explored different chunk reranking strategies to find optimal settings for each. While we have made significant improvements in retrieval relevancy and answer quality and accuracy, we believe, we still have more work to do to optimize the full pipeline.