RAG over a 500K-record game knowledge base
An AI assistant for World of Warcraft that answers from real Wowhead data — chat with a large structured base, with every number grounded in a retrieved source rather than the model's memory.
- Stack
- Python · LangChain · PostgreSQL · pgvector · FastAPI
- Scale
- 500K+ game records
- Retrieval
- hybrid — dense + BM25 + reranking
Structured game data is hostile to naive RAG
World of Warcraft data is structurally hostile to standard RAG. A query like "best way to farm Frostweave Cloth" requires cross-referencing item IDs, zone drop tables, mob respawn rates, and patch-version history — all in different schemas on Wowhead. There is no single passage that answers it.
A naive embedding search returns the most semantically similar text, not the most useful answer. It surfaces everything that mentions cloth farming and misses the specific drop table that actually matters. Ask it for a number and it gets worse: the model states a drop rate from memory, and it is confidently, precisely wrong.
Two-stage retrieval: resolve the entity, then search
We split retrieval into two stages. Stage one is entity resolution: the query is parsed to extract item names, zone names, and mechanics, then mapped to canonical IDs through a structured lookup. Stage two is semantic search over chunked strategy content, filtered by those resolved IDs as metadata constraints.
So "farm Frostweave" finds the correct item first, then retrieves only the guides and drop tables that reference that specific item — not everything that happens to mention cloth. Retrieval itself is hybrid: dense vectors and BM25 combined, then reranked, so exact-name matches and semantic matches both count toward the final context.
Every number is validated against its source
If the model generates a drop rate from memory rather than from retrieved data, the answer is wrong and stated with full confidence. So every numerical claim — drop percentage, respawn timer, required reputation level — is validated against the source chunk before the response is returned. A claim with no retrievable source is omitted or flagged as unverified, never smoothed over.
Answers cite the specific Wowhead page references, the patch version, and the data-collection timestamp behind each fact, so a user knows exactly how current the information is and where it came from.
Keeping the base current as the game changes
A game knowledge base is not static — every patch reshuffles drop rates, zones, and mechanics. The ingestion pipeline processes Wowhead data continuously, versions chunks when a patch changes them, and flags outdated content before it reaches retrieval, so an old answer cannot quietly resurface as a current one.
The shape of this generalizes. Any company sitting on a large structured base — catalogs, specs, policies, operational records — faces the same failure mode when it puts an LLM in front: fluent answers that are subtly wrong on the numbers, which is why the cost of a wrong answer decides how much checking a process needs. Two-stage retrieval, hybrid search, per-claim validation, and versioned ingestion are what turn "chat with your data" into answers a team can actually rely on — each one added because something simpler broke first, which we walk through in how a RAG knowledge base behaves at 500K records.
See what this would look like in your operations.
30 minutes · we map your 3 best automation opportunities · no obligation