Member-only story
Part 4: Chatting about company documents using RAG and Spring AI
A Step-by-Step Guide to Implementing RAG in Spring AI
In the previous parts of this tutorial, we learned several features of Spring AI. Our Employee Assistance chatbot can easily switch between different models (OpenAI, Llama, and DeepSeek) in part 2. We can also keep the chat history to get an answer from the model based on our previous questions and answers (part 1). We also learned how to write integration tests using Testcontainer and Ollama (part 3).
In this part, we want to expand our Employee Assistance chatbot and enable it to answer specific questions about the company's internal rules! To do so, we will use a famous technique called RAG.
· How to expand LLMs knowledge?
∘ What is the RAG technique?
∘ Difference between Chat history and Chat with document
· Implementing RAG using Spring AI
∘ Advisors API
∘ VectorStore API
∘ Embeddings Model API
∘ ETL Pipeline API
· Implementing Chat with document to Employee Chatbot using RAG
∘ Adding the ETL pipeline
∘ Embedding Model dependency
∘ Implementing RAG using the Advisor API
∘ Testing the result
· Final Thoughts
How to expand LLMs knowledge?
One of the biggest limitations of LLMs is that they are pretrained, which means they might lack information about certain domains or may have outdated information.
In our Employee chatbot,
Since our company rules are private and specific only to our company, our model (Llama) does not know about them. We need to expand its knowledge and provide context for it.
In part 1 of the tutorial, we learned about the Spring AI Advisors API and used the MessageChatMemoryAdvisor to provide our model with chat history.
MessageChatMemoryAdvisor works well for limited chat history, but this time, the use case is different, and the company rules document can be huge, so we can not send all of…
