Why I'm Adding RAG to My Sports Chatbot (and What Was Broken Without It)
When I started building Ball Knowers, I quickly learned that "knowing football" and "knowing today's football" are two completely different problems. I could ask my chatbot about Arsenal's "Invincibles" season or how many goals Thierry Henry scored, and it would answer confidently with useful historical context. But if I were to ask about how the latest signings for Liverpool were doing, how Manchester City performed last week compared to the week before, or even who won the latest Premier League season, the chatbot would immediately remind me that its knowledge only extended through October 2023.
Although I knew that the chatbot would need some configuration, I didn't know I'd encounter an issue with getting fresh context, the latest signings and roster updates for teams, fan sentiment, and team-specific updated knowledge. That was the moment that I realized my sports chatbot wasn't actually broken; it was missing memory. I thought I needed a better model. What I actually needed was Retrieval Augmented Generation (RAG).
What the Chatbot Could Already Do
When I began working on my Premier League chatbot, I wanted to build a tool that would keep track of all things Premier League, providing me with up-to-date information about my favorite teams, league standings, top performers, and just about everything that the many apps I was using could provide for me.
Ball Knowers already had a solid foundation. The application combined GPT-4o with a football data API, allowing users to ask natural-language questions while retrieving live information such as fixtures, league standings, top scorers, and upcoming matches. The application was built with Next.js, TypeScript, and OpenAI's GPT-4o model.
I was able to connect to an API that would provide up-to-date information that would be useful for my use case, with information such as upcoming games, so you could set your favorite team and see what they have coming up in terms of games. That API also provided up-to-date information for scores, standings, and league leaders in scoring and assists. I also had a chatbot functionality that could provide general football discussions with the prompt that I had given it.
// /lib/prompts.ts
export const instructions: string =
"You are an AI chatbot responsible for answering questions from users about
the English Premier League. Please refrain from answering any questions that
are not either soccer related or related to the English Premier League.";
This was perfect! I thought that the chatbot was at its best, then I asked a specific question in regards to current standings. Thats when I realized that I was only getting information that was up to the cutoff dates in October 2023, meaning that asking any current question would prove to be useless, as the chatbot itself couldn't help you with that.
I also couldn't get fan sentiment, real analysis on how a team was performing at the moment, or real insights that could help me with selecting players for my Fantasy lineup. That was the real thing that I wanted to solve.
A solution that I thought would work was function calling. Because I was already using the API to call for the top standings, league leaders in scoring, and assists, it would be easy to let the LLM call the proper function that makes the call to the API and get up-to-date information.
Here is the original architecture that I thought to build out for the work with function calling:
User
│
▼
GPT-4o
│
├── Football API
│
▼
Response
The addition of function calling was a great working solution for my use case. Asking "Who's leading the Premier League?" triggered a function call to the football API, which returned the latest standings before GPT generated a natural language response.
The Remaining Problem
Although function calling was, and still is, a good solution for getting current standings information or current league leaders in specific stats, the issue still remained: the LLM only had a knowledge base up until October 2023, meaning some questions that weren't retrieved from the API were still going to be difficult to solve.
A question like, "Who won the league last season?" or "How is Liverpool's new signing doing?" would cause this flow that I built out to fall apart.
The main issue was that the LLM had a way to get current data, but it still had no idea about anything that happened between October 2023 and yesterday, how fans felt about players, or even that players were on new teams. The API unfortunately couldn't help with this problem, as it could only give me the structured data that I had asked for, but it couldn't give me the answers to the questions I was really looking for.
In other words, I had built a chatbot that could retrieve facts, but it still couldn't understand the story surrounding those facts. That realization is what ultimately led me to RAG.
Adding Memory to the Chatbot
Well, firstly, what is RAG and what am I hoping to use it for?
Retrieval-Augmented Generation (RAG) is an AI technique that lets a large language model look up facts in an outside database or document before it answers questions. Instead of relying solely on the model's training data, RAG allows the chatbot to retrieve relevant documents, such as match reports, transfer news, and player updates, before generating a response. Rather than guessing, the model reasons over current information.
Here is what each term in RAG actually mean:
Retrieval: The system searches a private or live database for exact facts.
Augmentation: It adds those real facts into the AI prompt.
Generation: The AI writes clear answers using only those real facts.
This can be useful to a Sports Chatbot for many reasons but here are a few:
- Retrieve transfer news before answering transfer-related questions.
- Summarize recent match reports instead of relying on stale training data.
- Incorporate fan discussions to provide more contextual analysis.
- Ground responses in retrieved information rather than model memory.
- Cost Friendly: And I can't stress this enough, it is cheaper than retaining a whole new AI model every day, especially one that is costly
Why not choose other options?
At first I considered simply switching to a newer model or fine-tuning one with football data. But neither approach solved the real problem.
Football changes every day. Fine-tuning a model every time a transfer happened or a match ended would be expensive, slow, and difficult to maintain.
What I needed wasn't a smarter model, it was a way for the model to retrieve fresh information whenever a user asked a question.
How I'm Building the RAG system
So now that we know what the problem is, I need to establish an architecture that gives the chatbot access to up-to-date Premier League knowledge without switching to a newer model.
Instead of asking the LLM to "remember" everything that's happened in football over the past few years, and what's happening every day as well, I am changing the responsibility of the chatbot. The chatbot will no longer be the source of truth; it will now be the reasoning engine. Its job will be to retrieve the most relevant information and then generate a response based on that reasoning.
The architecture that I'm aiming to build:
Football APIs
News Articles
Match Reports
Transfer Updates
Fan Discussions
│
▼
Text Processing
(Cleaning & Chunking)
│
▼
OpenAI Embeddings
│
▼
Supabase pgvector
(Vector Store)
│
▼
Semantic Similarity Search
│
▼
Relevant Documents
│
▼
GPT-4o
│
▼
Final Response
The first step in this architecture is collecting information. This step isn't necessarily new to the process, as the existing football API has been responsible for giving up-to-date league standings and player statistics. The difference here is that, along with the API, I will be expanding the knowledge base by ingesting less structured information such as transfer news, match recaps, and plenty of football news for the Premier League. These are the types of information that can provide context that the API could not provide by itself.
After the data has been collected, it has to be cleaned and split into smaller chunks, and then converted into vector embeddings using OpenAI's embedding model. These embeddings will then be stored in a vector database, allowing for semantic searches instead of keyword matching.
After that, the idea will be that the app won't immediately send the question straight to the LLM. Instead, it'll first search through its own knowledge base, looking for the most relevant pieces of information related to that prompt.
For example, if someone is to ask about:
"Who's top of the Premier League?"
"When does Arsenal play next?"
"Who leads the league in assists?"
Those questions are best answered by calling the football API directly because the data is structured and changes frequently.
On the other hand, questions like:
"Why are Chelsea fans excited about their new manager?"
"How has Manchester United's midfield improved this season?"
"What's the reaction to Liverpool's latest signing?"
Those questions are best answered with contextual information, and the vector database will be a perfect place to search before reaching the LLM.
Function calling was definitely a great first step in solving this issue, but with the combination of RAG and function calling, the chatbot can now provide good responses to different questions. Function calling will still handle the information that needs real-time data, while RAG will provide the rich context needed for more specific conversations.
What RAG will Improve
Now that I have explained a bit about RAG, this is what I'm expecting to solve with RAG integration into my system.
Without RAG
User
↓
LLM
↓
Training Data (2023)
Answer
With RAG
User
│
▼
Retriever
│
──────────────
Football News
Transfer News
Match Reports
Fan Discussions
──────────────
│
▼
Relevant Documents
│
▼
GPT-4o
│
▼
Response
Since the 2026/27 Premier League season has not begun yet, but we are in the summer transfer market, I'm especially hoping to get player updates, new club signings, and confirmed signing data.
I'm hoping to get more fan sentiment as well, since fans like to stay connected, and this is the best way to give the AI some insight into what the fans are thinking.
I'm also hoping to give it up-to-date context so that the AI doesn't trip up on questions like "Who are the current Premier League reigning champs?", but instead can go beyond that and answer questions such as "What was the score for the Liverpool game yesterday?" Hopefully, with RAG, these issues can and will be solved.
Building a sports chatbot taught me a lesson that applies to many AI products that AI product builders should also keep in mind when building: users don't judge intelligence by how much a model knows; they judge it by whether it knows what happened yesterday.
You can use function-calling tools that can help to generate an AI response, but RAG will give you the additional context that is necessary.
The next version of Ball Knowers won't just answer "Who is Tottenham's right winger?" It should answer "Why are Tottenham fans arguing about why the right winger is the best player after last night's match?"
That's the gap I'm trying to close, and that's why RAG is becoming the most important feature I'm adding to the project.