Healthcare RAG & Clinical NLP System

Hospital clinicians spend a surprising amount of their day looking things up. Not diagnosing, not treating — looking things up. The exact dose of an anticoagulant for a patient with renal impairment. The current institutional guideline for sepsis bundles. Whether a specific drug interacts with a specific chemotherapy regimen at a clinically meaningful level. Whether the last version of the antibiotic stewardship policy still recommends the same first-line agent for a community-acquired pneumonia in a patient over 65. The answers exist. They live in PDFs on an intranet, in a formulary system, in a stewardship wiki, and in a set of monographs licensed from a third party. None of those systems talk to each other, and search inside each of them is, being generous, uneven.

The ask I received was framed as 'a ChatGPT for our clinicians,' which is exactly the framing you should push back on politely and firmly. What clinicians needed was not a general chat interface. They needed a fast, boring, cite-backed lookup service that returned the specific paragraph of the specific policy or monograph that answered the question, along with a link to open the source document at that exact section. Fluent prose was not the value. Trustable retrieval and a hard citation were the value. If the system could not find a good answer, the correct behavior was to say so and route the clinician to a human — a pharmacist, a specialist, an on-call attending — not to synthesize a plausible-sounding paragraph from partial matches.

The corpus was around 180,000 documents: institutional guidelines, order sets, formulary monographs, a licensed drug reference, de-identified teaching case notes, and a set of externally sourced clinical practice guidelines from major societies. Sizes and formats varied wildly. Some guidelines were clean 30-page PDFs with a table of contents. Others were scanned faxes from a decade ago, OCR'd once and never checked. Case notes were structured HL7-derived JSON that had already been de-identified through a separate pipeline I did not own. Access rules were coarser than in the enterprise RAG project — most content was visible to any credentialed clinician — but the compliance surface was larger. Anything that touched PHI, even indirectly, had to be logged, and any query that looked like a request for individualized patient care advice had to be refused with a clear message that the system was a reference tool, not a decision-support tool.

Latency mattered here even more than in a typical enterprise RAG deployment. The primary usage pattern was a clinician standing outside a patient's room with a workstation-on-wheels, or on a phone during a code, needing an answer in seconds. My target was two seconds to first token and under five seconds to a full cited answer at p95. If the system was slower than opening the PDF manually would have been, adoption would collapse and nobody would tell me — they would just stop using it. Silent abandonment is the failure mode you never see coming, and I designed against it explicitly.

The compliance constraints were the frame the rest of the system had to fit inside. All PHI stayed inside the network's own tenant. No document content and no query text left the environment except through an approved model endpoint that had a signed BAA. Every query was logged with the authenticated user, the timestamp, the retrieved chunk identifiers, the generated answer, and the sources cited. Logs were retained for the retention window compliance required and were queryable by the audit team without touching the application database. If a clinician later reported a wrong answer, I had to be able to reconstruct exactly what the system saw and what it produced. This audit-first posture drove several downstream decisions, most notably the decision to make chunk identifiers content-addressed and version-stable.

The refusal boundary was the other constraint I treated as sacred. The system was allowed to answer questions about guidelines, monographs, and reference material. It was not allowed to answer questions of the form 'my patient is a 68-year-old with X and Y, should I do Z.' Those are decision-support questions, and decision-support in a clinical setting is a regulated activity with a different bar of evidence than a reference lookup. I built a classifier upstream of retrieval that flagged patient-specific phrasing and either rewrote the query into a generic reference question or, when that was not possible, returned a scripted refusal that pointed the clinician to the appropriate consult service. False positives from that classifier were annoying but recoverable. False negatives were not.

The non-goals were also load-bearing. I did not build a summarization surface for individual patient charts, even though people asked for it. I did not build a differential-diagnosis feature. I did not build anything that could be construed as generating clinical recommendations. I did build a set of narrow, well-scoped capabilities that added up to a fast reference tool, and I resisted every attempt to extend the scope in the first six months. Scope discipline is the difference between a clinical AI project that ships and one that lives in pilot forever, and I have watched enough of the second kind to know which side to err on.

The hot path is intentionally shaped like the enterprise RAG system I had built before, because that shape works, but with three domain-specific twists. A query enters through an authenticated FastAPI service running inside the hospital network. The service resolves the clinician's identity, role, and specialty into a set of retrieval filters — a cardiology fellow gets cardiology-tagged guidelines boosted, but not exclusively — and passes the query through the patient-specificity classifier. If the classifier flags patient-specific phrasing, the query is either rewritten or refused. If it passes, the query goes through a synonym expansion layer backed by a small UMLS-derived thesaurus that maps common lay terms and abbreviations to their canonical clinical forms. 'MI' becomes 'myocardial infarction OR MI.' 'GI bleed' picks up 'gastrointestinal hemorrhage.' This expansion happens on the sparse retrieval branch, not the dense one, because dense embeddings already handle most synonymy and expanding them tends to blur the vector rather than sharpen it.

Retrieval is hybrid — dense over a self-hosted vector store and sparse over OpenSearch — and results are fused with reciprocal rank fusion, then reranked by a cross-encoder I fine-tuned on labeled clinician query-chunk pairs. The reranker was the single largest quality lever, moving nDCG at 10 by roughly 9 points on our held-out set compared to fused retrieval alone. Generation is a call to an approved LLM endpoint with a strict system prompt that requires either an answer grounded in the provided chunks with inline citations to chunk IDs, or an explicit refusal. Citations are rendered in the UI as clickable references that open the source PDF at the correct section using a pre-computed page anchor.

The offline pipelines are where most of the domain-specific engineering lives. Ingestion for guidelines uses a section-aware chunker that respects the outline structure of clinical practice guidelines: recommendation blocks, evidence summaries, and background sections are chunked as coherent units rather than sliced at arbitrary token boundaries. Every chunk carries metadata for guideline title, section, recommendation strength if present, evidence level, and publication date. Ingestion for monographs uses a schema-aware parser because monographs are semi-structured — indications, contraindications, dosing, adverse effects — and preserving that structure lets me filter and boost by section at retrieval time. Ingestion for teaching case notes runs a second-pass de-identification check even though the upstream pipeline already de-identified them, because trust-but-verify is cheap and the cost of a leak is not.

Chunk identifiers are a content hash plus a document-version tag. When a guideline is updated, old chunks are retained in the index but marked superseded, and new chunks are added with a new version tag. Retrieval filters superseded chunks out of the default view but the audit system can still resolve a citation from six months ago against the exact chunk revision that generated it. This is the piece of the system that took the longest to design and gets the least attention in demos, and I would build it the same way again.

The section-aware chunker was worth every hour I spent on it. Naive fixed-window chunking on clinical guidelines produces chunks that split a recommendation from its evidence grading, or a dosing table from the drug it applies to, and the model then either invents the missing context or hedges its way into unhelpfulness. Section-aware chunking keeps each recommendation, its rationale, and its evidence summary together as a single unit, and it keeps dosing tables intact. On a bench of 400 curated clinician questions, moving from fixed-window to section-aware chunking improved answer-with-correct-citation rate by about 14 points, larger than any single model change I made.

Synonym expansion on the sparse branch was a smaller but reliable win. Clinicians ask questions with abbreviations, brand names, and lay phrasing interchangeably, sometimes within the same sentence. A pure BM25 index treats 'acetaminophen' and 'paracetamol' as unrelated tokens. Expanding through a small hand-curated thesaurus, layered on top of a UMLS-derived synonym set, closed that gap without introducing the noise you get from full query expansion via an LLM. I tried LLM-based query rewriting early and abandoned it: it was slower, less predictable, and occasionally produced expansions that shifted the intent of the question in subtle ways.

Metadata filters at retrieval time were the third lever. When a clinician asks a question tagged as pediatric, boosting pediatric-scoped guidelines and monographs, and demoting adult-only content, made results dramatically more relevant without any change to the underlying index. The specialty and patient-population tags on chunks were maintained as part of ingestion, not inferred at query time, because inference at query time is a per-query cost you pay forever and inference at ingest time is a fixed cost you pay once.

One retrieval technique I evaluated and rejected was hypothetical document embedding, where you generate a fake ideal answer with an LLM and embed that instead of the query. It worked well on paraphrased conceptual questions and worse on specific factual ones, which is the opposite of what I needed. Clinical questions skew concrete: exact drug, exact dose, exact indication. I stayed with straight query embedding and moved the quality budget to reranking and chunking, where it paid back more reliably.

The system prompt for the generator is short and does most of the safety work. It instructs the model to answer using only the numbered chunks provided, to cite the chunk numbers inline in square brackets, to refuse if the provided chunks do not clearly answer the question, and to never generate content that reads as individualized clinical advice. The prompt names specific patterns it must refuse: dosing recommendations for a described patient, diagnostic suggestions, and any answer that would require weighing patient-specific factors the system cannot see. The refusal message is scripted and points to the appropriate consult service based on the query's specialty tag.

The refusal-first posture matters more than any single prompt-engineering trick. A generator that is willing to refuse can be trusted to answer. A generator that never refuses is one that will fabricate the answer to a question the retrieval layer failed to serve. I set the retrieval confidence threshold for refusal deliberately conservatively during the first six weeks and eased it slowly as I built up evidence from the audit logs about where refusals were correct and where they were over-cautious. The bias I optimized for was 'refuse and be slightly annoying' over 'answer and be occasionally wrong.' In a clinical setting, that trade is not close.

Answers are rendered with the citations inline and the source chunks displayed in a side panel with a direct link to open the PDF at the correct page. Clinicians rarely read past the first citation, and that is fine — the citation is the point. The prose is a wrapper around the citation. I resisted every design suggestion to make the answer longer or more elaborate. The best answer is the shortest one that resolves the question, followed by a link the clinician can click if they want to read more.

The evaluation harness was built before the second week of feature work, and it is the reason the project shipped. I curated an initial 400-question golden set with the help of two clinical champions per specialty. Each entry contained the question, the expected source document, the expected section or chunk, and a short rubric for what a correct answer must contain. I ran the full retrieval and generation pipeline against the golden set nightly and tracked four metrics: retrieval recall at 10, correct-source-cited rate, answer-contains-required-facts rate, and refusal rate on the subset intentionally seeded with unanswerable questions. Any change to the pipeline had to move at least one of those numbers up without moving the others down more than a small margin, or the change was reverted.

The golden set grew over time from real usage. When a clinician flagged an answer as wrong, that question and the expected answer became a new golden-set entry. When we shipped support for a new specialty, we added a hundred new entries curated by that specialty's champion. The golden set is now around 1,300 entries and it is the single most valuable artifact the project produces. If I had to hand the whole system over to a new team tomorrow and could only leave them one thing, it would be the golden set, because that is the thing that lets them change anything else safely.

I also ran a separate adversarial suite of questions designed to trip the refusal boundary: patient-specific dosing questions, diagnostic prompts, questions that mixed a legitimate reference lookup with a decision-support ask. The refusal classifier was tuned against that suite, and any change that reduced adversarial refusal rate was blocked from release regardless of what it did to other metrics. That was one of the few hard gates in the CI pipeline, and it caught two regressions during the pilot.

The first meaningful incident was a stale-guideline problem. A national guideline had been updated, our ingestion pipeline had pulled the new version, but a caching layer in front of the retriever was still serving results scored against the previous version's embeddings for about six hours. Clinicians got answers that cited the old recommendation. The answers were internally consistent — the cited chunk did say what the answer claimed — but the cited chunk was superseded. The fix was to key the retrieval cache on the guideline version tag, not just the query, so any version bump invalidated the relevant cache entries automatically. The lesson was to treat 'the source has changed' as a first-class event, not something you notice at the next full reindex.

The second incident was a subtle prompt-injection through document content. A monograph from a third-party source contained a footnote with instruction-shaped text that a previous editor had left in as a template placeholder. The generator, when that footnote was in the context window, occasionally produced responses that echoed some of the placeholder text. It never produced anything clinically wrong, but the responses looked strange to reviewers. I added a content-filter step in the ingestion pipeline that flags instruction-shaped patterns in source documents and routes them for review before they enter the index. It is a coarse filter and it produces false positives, but the review queue is small enough that a librarian can clear it in an hour a week.

The third incident was model-endpoint drift. The approved LLM endpoint pushed a minor version update that slightly changed the model's willingness to refuse. Refusal rate on the adversarial suite dropped by three points overnight. Nightly evals caught it. I rolled the endpoint back to the pinned version and coordinated with the vendor on the update timing. That incident is the reason my current default posture is to pin to specific model versions everywhere the vendor allows it, and to run a canary comparison against a golden set on any version bump.

The fourth is ongoing rather than an incident: user education. Clinicians who came from consumer chat products expected the system to answer patient-specific questions and were mildly frustrated when it refused. The fix is not a code fix. It is training material, in-product microcopy, and a clear message about what the system is for. Every refusal now includes a one-line explanation and a link to the appropriate consult service. Complaints about refusals dropped substantially once the microcopy landed. The lesson I keep learning is that in a clinical product, the interaction design around a refusal is as important as the refusal logic itself.

The pilot ran across three specialties for four months. Median time from question to cited answer, measured from the moment the clinician started typing to the moment they clicked through to the source, dropped from around four minutes on the legacy intranet search to under twenty seconds on the new system. That number is the one that made the project's case internally, because it is easy to explain and easy to verify.

Adoption inside the pilot cohort reached about 72% weekly active use by the end of month two and held there. Answer-with-correct-citation rate on the golden set was 91% at go-live and 94% by the end of the pilot after two rounds of retrieval tuning. Refusal rate on the adversarial suite was 100% at go-live and stayed there through the pilot. There were zero incidents where the system produced individualized clinical advice, and I would rather that number stay boring forever than have an exciting story to tell.

The audit system logged every query and every citation and was queried in anger twice: once by a compliance review that wanted to see the distribution of refusals by specialty, and once by a physician who wanted to know exactly what the system had said in response to a question their colleague had asked earlier in the week. Both queries were answered in under an hour from stored logs. The audit trail existing and being trivially queryable was worth more than the sum of every clever piece of the retrieval stack.

I would build the refusal classifier as its own first-class model with its own eval suite, from day one, instead of starting it as a set of regex-and-rules on top of the query classifier. It grew into a first-class model eventually because it had to, and the retrofit was more work than starting there.

I would invest earlier in the source-document review workflow. The instruction-shaped footnote incident was a symptom of a broader class of ingestion-quality problems, and a small librarian-facing review UI would have caught several smaller issues that I only noticed later.

I would resist the summarization asks even harder than I did. I said no to summarization of patient charts, which was the right call, but I initially said yes to summarization of guideline sections as a convenience feature. Every complaint I got about answer quality traced back to that surface. When the system quoted the guideline verbatim with a citation, clinicians trusted it. When it summarized, they questioned it, even when the summary was accurate. In a clinical reference tool, the primary source is the product. Summaries are a subtraction, not an addition.

And I would have written the incident-response runbook before the pilot, not after the first incident. The runbook for 'the system produced a wrong answer that a clinician acted on' is a document you want to have already discussed with your compliance and clinical safety partners before you need it, and writing it is the fastest way to surface the disagreements you did not know you had.