Translations:FACTS About Building Retrieval Augmented Generation-based Chatbots/67/en

Our work can be compared with RAG papers on various topics dealing with RAG quality along all the FACTS dimensions we presented (freshness, architecture, costs, testing, and security). Due to lack of space, we contrast our work with selective works. Barnett et. al. (3) presented seven failure points when engineering RAG systems. In their work, they highlight the challenges of getting retrieval augmented generation right by presenting their findings from having built three chatbots. Wenqi Glantz (6) elaborated 12 RAG pain points and presented solutions. We experienced similar challenges first-hand when building our chatbots. However, none of these works discuss the challenges with complex queries, testing, dealing with document security, and the need for flexible architectures. In our work, we not only build on failure/pain points of RAGs as mentioned above, but also present our 15 control points in RAG pipelines and offer specific solutions for each stage. Also, we extend our insights and present practical techniques for handling complex queries, testing, and security. We present a reference architecture for one of the implementations of agentic architectures for complex query handling, strategies for testing and evaluating subjective query responses, and raised awareness for dealing with document ACLs and security. Furthermore, we present a reference architecture for a flexible generative-AI based Chatbot platform.