Exploring how Retrieval Augmented Generation and NVIDIA NIM can transform resource accessibility for students
Author(s)
Designation
Date
Prakruti Srikanth, Veer Guda
October 9, 2025
AI@GT Projects: More Than Just Hackathons
AI@GT is on a mission to make students more involved with AI, and a large part of that is in the projects that our members do. These are student led initiatives that aim to create real world impact. Today, we have a spotlight on the RAG Chatbot for GT students.
What the project is about, what was it designed to solve?
As students, our resources tend to be very scattered, which can lead to wasted time and frustration. I’m sure many are familiar with the exasperation that comes with trying to find the answer for 1 question, then being redirected several times, only to not have the question answered. Student and AI@GT member Veer Guda was all too familiar with that. That’s why last year, when it came time to pick projects, he decided to rectify this by creating a RAG Chatbot for GT students.
Key Features of the Chatbot
RAG stands for retrieval augmented generation, and it’s an AI chatbot that combines an LLM with an external knowledge base (in this case, that knowledge base is a collection of GT resources). What makes this advantageous as compared to a traditional LLM is that it keeps relevant up-to-date information from external sources rather than just going based on the data it was trained on. This way, it can provide more accurate and specific answers to people’s questions. This type of AI model was the ideal choice to give specific information that’s relevant to GT students. AI@GT partnered with NVIDIA to make this project happen, by using their NIM (NVIDIA Inference Microservices). The NIM is an optimized software container that expedites the process of deploying AI models across different environments.
Team Info
Building something this technically detailed was no easy task. This team had to split the work into 2 subdivisions: A data processing side and an LLM pipeline side. The data processing side worked on gathering all requisite data and formatting it for easy retrieval in the vector database. The LLM pipeline side focused on building out the RAG system and ensuring that the query, to processing, to output pipeline worked properly.
Technical Design
Both of these tasks were immense, considering that in total, the team had to process over 1 million records from Georgia Tech websites. They did this using Scrapy and BeautifulSoup. Scrapy was used to webscrape (extract information and data from websites), while BeautifulSoup was used to parse HTML and XML documents (analyze the information). Both of these worked in tandem to transform the data into structured markdown JSON files. After that, they designed a context-aware chunking and embedding pipeline with LLaMA 3.2b embed-qa to enhance retrieval efficiency and storage optimization. This divided large documents into smaller and more meaningful chunks, representing them numerically (as embeddings), which made finding relevant information faster and more accurate. Finally, they deployed a reranking system using NVIDIA NIM, which refines and reorders an initial set of search results to improve relevance and quality.
Lessons/future improvements and impacts
In the end, after months of hard work the team put in, the RAG GT Chatbot was born. As mentioned earlier, the knowledge base of this chatbot is constantly updated through the RAG system, meaning the chatbot always has relevant, accurate, and up to date information. Beyond this, the chatbot has built in functions to prevent abuse, ensuring that only GT-specific questions are asked. This helps make sure that the servers aren’t overwhelmed by non-GT students asking general questions. The chatbot is able to provide reliable, context-aware responses with sub 2-second latency. It’s also scalable, and able to handle appropriate levels of concurrency for campus usage. This hard working team’s work isn’t yet done, and even with this project, they can still find room for improvement. One point of contention they stated was resource efficiency. They built separate scrapers for each domain, which was time consuming and suboptimal. However, this is just the first step, and they managed to build something that utilized relatively new technology (NIM) to create something genuinely useful for GT students. At AI@GT, we strive to make a real impact and foster the talent that will change the world. This project truly embodies our goal: to build meaningful AI based projects that will help people. If you want to keep up with AI@GT and find out about more projects like this one, you can check out our instagram and substack.
