M.S. by Research — neural information retrieval
Teaching a retriever what a bug report is really about
IIIT Hyderabad · May 2016 — Aug 2018
Dense retrieval for short, noisy documents: a BERT model trained with triplet learning and inter-document context, and a screening system that saved a third of reviewer time at full recall.
- screening time saved at 100% recall
31%
screening time saved at 100% recall
- Recall@1 across three corpora
61–65%
Recall@1 across three corpora
- recall improvement over baselines
10–15%
recall improvement over baselines
The problem
Short documents — bug reports, abstracts — give a retriever almost nothing to work with. Lexical overlap is thin and often misleading, because the words that repeat are the generic ones.
In systematic review screening, recall is not negotiable. A system that misses one relevant paper is not a faster system; it is a wrong one.
Duplicate bug detection has the same shape: near-identical reports use entirely different vocabulary, and the signal lives in what surrounds a report rather than in the report alone.
Constraints that shaped it
- 100% recall is the floor for the screening system. Time saved is the only variable.
- Evaluation across three independent corpora — Mozilla, Eclipse, NetBeans — to make sure the gain was not one project's quirk.
Architecture
Select any box for the design decision behind it.
What I owned
- 01The full thesis: problem formulation, model, training regime, and evaluation.
- 02NITBUG — the BERT-based dense retrieval model with triplet learning and inter-document context.
- 03The PubMed screening system evaluated at 100% recall.
What I would do differently
Everything I do now with RAG is this problem wearing different clothes. Retrieval quality sets the ceiling; the generator only decides how close you get to it. Eight years later that is still the part teams under-invest in.