What this system does.
Universities and colleges sit on years of knowledge: syllabi, academic policies, handbooks and archived research. Finding a precise, trustworthy answer in that material is slow, and general-purpose chatbots make it worse by inventing sources that do not exist.
This system is a retrieval-augmented generation (RAG) layer built specifically for that problem. Every answer is grounded in the institution's own documents, every claim carries a citation back to its source, and when the evidence is not strong enough the system says so instead of guessing.
Why the existing setup was not good enough.
Academic teams needed answers grounded in their own documents. Off-the-shelf chatbots either hallucinated citations or dropped context across multi-step questions, and faculty needed answers they could trust enough to publish in a course handbook.
How I solved it.
Hybrid retrieval (BM25 plus dense embeddings), cross-encoder reranking, then a constrained generator that refuses to answer when grounded confidence drops below a threshold. State is managed by a LangGraph state machine with checkpointing for replay and audit.
How it is built.
- 01Ingestion → chunking on semantic boundaries → OpenAI embeddings → pgvector
- 02Hybrid retriever combines lexical (BM25) and dense recall, top-k = 20
- 03Cross-encoder reranker narrows the candidates to the top 8 with calibrated scores
- 04Generator runs inside a LangGraph graph with a deterministic refusal node
- 05Postgres-backed checkpointer stores conversation state for replay and audit
The engineering choices that mattered.
Hybrid retrieval instead of vectors alone
Academic documents are full of exact identifiers such as course codes, policy numbers and proper names, which dense embeddings tend to blur. BM25 catches those exact matches while dense retrieval catches paraphrased questions, so combining both improves recall on both kinds of query.
A cross-encoder reranker before generation
Retrieval is tuned for recall (20 candidates). The cross-encoder reads each question and passage together and keeps only the 8 most relevant, so the generator sees less noise and cites the right passage.
Refusal is a node in the graph, not a prompt instruction
Asking a model nicely not to hallucinate is not a guarantee. The LangGraph graph has a deterministic refusal path that runs when grounded confidence drops below a threshold, so a weak answer is never shown as a confident one.
pgvector inside Postgres
Embeddings live next to document metadata and the conversation checkpoints in one Postgres database, which keeps filtering, joins, backups and access control in a single system.
Checkpointed state for replay and audit
Every conversation step is persisted. Any answer can be replayed to see exactly which passages were retrieved and why, which matters when the answer ends up in a course handbook.