Ask 40 NLP papers a question
This is a live demo of sourcebound, an open-source tool that answers questions over documents. It searches the papers, a language model writes an answer that cites numbered passages, and a small model, openjev-flash, checks each cited sentence against the passages it cites.
Privacy: your question and the passages found for it are sent to free language-model providers through OpenRouter, which may log them. Do not type anything private.
Question
supported the sentence is entailed by the passages it cites (openjev-flash, P(entailment) ≥ 0.5). not supported it is not. This is one small model's judgement, not a guarantee.
Saved answers
Recorded in advance with the same pipeline and models. They are shown when the live demo is over its limit or unavailable, and you can open them any time.
How it works
What runs where
- Your browser loads this page from a Cloudflare Worker. The Worker checks a bot challenge when one is configured, allows 10 questions per visitor per hour and 950 per day in total, and forwards the question (and nothing else) with a secret header.
- A small home server (a 4-thread VM, no GPU) reached through a Cloudflare Tunnel runs the real Python sourcebound. It answers at most two questions at a time.
- Retrieval: the question is embedded with
BAAI/bge-small-en-v1.5and searched in ChromaDB (vector, with MMR) and in SQLite FTS5 (BM25). The two lists are fused with reciprocal rank fusion. openjev-flash, run as an int8 ONNX model on the CPU, reranks the top 10 candidates, and the best 5 passages go to the model. - Answer: an LLM from OpenRouter's free tier writes the answer from those passages (
nvidia/nemotron-3.5-super-vl:free, thennvidia/nemotron-3-ultra-550b-a55b:free, thengoogle/gemma-4-26b-a4b-it:freeif the previous one is rate limited or fails). Every[n]is checked against the passages the model was shown, and numbers it was not given are removed. - Check: each answer sentence with a citation is scored by openjev-flash against the text of the passages it cites. The badge shows that result.
The index holds 1,396 chunks from 40 Qasper validation papers. It has no knowledge graph. Questions are answered over all 40 papers at once, so name the topic or paper in the question. If nothing relevant is found, sourcebound says so without calling the model.
Evaluation numbers (copied from the README, unchanged)
From the repository's evaluation on the same 40 papers: 176 answerable questions, cells are the mean with a 95% bootstrap interval. The demo uses hybrid retrieval with rerank, but over 10 candidates and with the models listed above, so these tables are not a measurement of this exact deployment. See the README for what they do and do not support.
Retrieval, within the question's paper
| setting | recall@5 | recall@10 | MRR@10 |
|---|---|---|---|
| vector | 0.546 (0.477-0.611) | 0.761 (0.704-0.813) | 0.432 (0.377-0.487) |
| vector + MMR | 0.510 (0.440-0.580) | 0.719 (0.659-0.775) | 0.418 (0.362-0.476) |
| BM25 | 0.496 (0.430-0.564) | 0.710 (0.648-0.767) | 0.374 (0.319-0.430) |
| hybrid (vector + MMR, BM25, RRF) | 0.586 (0.521-0.652) | 0.752 (0.696-0.805) | 0.450 (0.397-0.504) |
| hybrid + graph | 0.562 (0.494-0.627) | 0.740 (0.680-0.795) | 0.409 (0.358-0.463) |
| hybrid + rerank | 0.629 (0.566-0.696) | 0.801 (0.750-0.851) | 0.449 (0.397-0.503) |
| hybrid + graph + rerank | 0.633 (0.568-0.698) | 0.810 (0.760-0.857) | 0.450 (0.398-0.505) |
Retrieval, pooled over all 40 papers
| setting | recall@5 | recall@10 | MRR@10 |
|---|---|---|---|
| vector | 0.243 (0.183-0.304) | 0.355 (0.289-0.422) | 0.181 (0.137-0.226) |
| vector + MMR | 0.201 (0.144-0.260) | 0.289 (0.228-0.356) | 0.156 (0.113-0.202) |
| BM25 | 0.255 (0.194-0.320) | 0.343 (0.273-0.411) | 0.174 (0.130-0.221) |
| hybrid | 0.331 (0.267-0.399) | 0.399 (0.331-0.466) | 0.205 (0.158-0.252) |
| hybrid + graph | 0.284 (0.220-0.350) | 0.405 (0.338-0.474) | 0.193 (0.146-0.239) |
| hybrid + rerank | 0.329 (0.264-0.397) | 0.427 (0.358-0.499) | 0.236 (0.185-0.287) |
| hybrid + graph + rerank | 0.328 (0.263-0.397) | 0.418 (0.349-0.487) | 0.232 (0.181-0.283) |
Answer quality (30 questions, 15 for agent mode)
| run | mode | answered / no answer | cites something | citation precision | cited sentences judged supported |
|---|---|---|---|---|---|
| bonsai-2 (Ollama) | pipeline | 28 / 2 | 24 | 0.348 (16/46 cited passages overlap gold); per-answer mean 0.431 (0.257-0.604) | 0.963 (0.907-1.000) of 54 |
| bonsai-2 (Ollama) | agent | 15 / 0 | 14 | 0.214 (6/28); per-answer mean 0.274 (0.107-0.476) | 0.738 (0.595-0.857) of 42 |
| nvidia/nemotron-3.5-super-vl:free (OpenRouter) | pipeline | 29 / 0 (1 call failed) | 25 | 0.346 (18/52); per-answer mean 0.448 (0.301-0.601) | 0.909 (0.836-0.982) of 55 |
Citation precision is the share of cited passages that overlap annotated gold evidence (a lower bound). Faithfulness is the share of cited answer sentences that openjev-flash scores as entailed (probability ≥ 0.5).