Radiology AI
Chest X-ray triage, explainability and semantic search — one shared model across three workflows.
14 conditions · 112k X-rays · 3 workflows on 1 model
The problem
Three different things a radiology team wants from an X-ray model — triage, region explanation, and similar-case retrieval — are usually built as three different systems. They don't have to be. This platform runs one DenseNet for triage and explainability, plus CLIP for retrieval, on the NIH ChestX-ray14 corpus (112,120 frontal images, 14 disease labels).
Who this is for
Medical AI engineers, clinical informatics teams, anyone evaluating how a multi-label classifier should be wired into a real workflow.
Architecture
- DenseNet121 (torchxrayvision)
- Multi-label classifier loaded once at boot, shared between triage and explainability.
- Class Activation Maps
- Computed from the DenseNet's final feature maps + classification weights — no extra model.
- CLIP ViT-B/32 (open_clip)
- Zero-shot text→image and image→image retrieval, plus auto-generated tags.
- FHIR-style context
- Patient + imaging metadata wrapped in a FHIR shape so the demo speaks the same vocabulary clinical systems do.
- FastAPI + Next.js
- Backend on port 10001, frontend on 10000; tunnel routes radiology.djkimlab.com to the frontend.
Request / data flow
- 01Image uploaded → DenseNet runs once → 14 per-condition probabilities.
- 02User clicks a condition → CAM derived from that class's weighted feature map → heatmap overlay.
- 03User flips to search → CLIP embeds the query (text or image) → cosine over pre-computed library embeddings → ranked hits.
Key decisions
Use torchxrayvision DenseNet121 rather than training from scratch.
whyChestX-ray14 is a known benchmark; the value here is the workflow integration, not yet-another model card.
Three workflows on one shared model.
whyTriage and explainability are literally the same forward pass; splitting them was a code smell.
FHIR context, not raw fields.
whyAnyone working in a hospital IT environment immediately knows what a Patient + Imaging Study looks like; that's the integration story.
Stack
If I rebuilt it
- ›Bring up image-level uncertainty (calibration / temperature scaling) so triage shows confidence bands.
- ›Cache CLIP embeddings on disk per dataset version, not in memory at process start.