Archetype 12
Diagnostic Hierarchy CLIOH
given a patient's presenting features, produce a feature-conditional ordering of the differential diagnosis — which condition is most likely, second, third — as a decision-support aid.
- Evidence standard
- external — validated against the confirmed final diagnosis
- What you compare
- Presenting features → candidate conditions (the worth is *conditional on the feature profile*)
- Panel
- Diagnosticians (clinicians who actually work up these presentations)
- Output
- A posterior DDx ordering (ranked differential), optionally an EHR/decision-support widget
- Validation
- Top-1 / top-3 accuracy and ROC/AUC against final dx; drift monitoring
What it's for
A CLIOH study in which diagnosticians make pairwise judgments of which condition is more likely given a feature profile, and a covariate-structured Bradley–Terry model produces a feature-conditional posterior ordering of the differential diagnosis.
"Given this presentation — this age, this fever, these labs, this exam — what is the most likely diagnosis, and what's the rank order of the rest?"
When to use it
(a) The differential is well-bounded (a known, finite set of competing conditions); (b) presenting features meaningfully shift the relative likelihoods; (c) trainees/covering staff order the differential inconsistently and a normative aid would help; (d) a confirmed final-diagnosis dataset exists or can be assembled for validation; (e) the clinical stakes of mis-ordering justify the build.
When not to
(a) You're assigning a single case to one categorical grade, not ordering competing conditions → Classification §3; (b) you're producing an urgency score for prioritization, not a likelihood ordering → Triage §4; (c) you're weighting predictors for one outcome → Risk-Factor §7; (d) the differential is effectively open-ended (too many rare conditions to pair meaningfully); (e) a validated diagnostic rule already performs well locally (use it).
What you get
A posterior DDx ordering for a given feature profile — and, if deployed, an EHR/decision-support widget that takes the work-up data and returns the ranked differential with confidence. Not a clinical decision tool until separately IRB-reviewed, prospectively validated, and operated under an SaMD-grade quality management system (binding caveat — the DR is explicit that Diagnostic CLIOH toward live decision support requires this). The instrument UI makes no unverified diagnostic or regulatory claim.
A real example
- Pediatric limp DDx for ED decision support (candidate first instance): order septic arthritis vs transient synovitis vs Perthes vs SCFE vs malignancy/other, conditional on age, fever, weight-bearing status, and inflammatory markers — the classic high-stakes pediatric-orthopaedic differential, where existing clinical prediction rules show the appetite for a structured aid. The reusable artifact is the feature-conditional case bank + validated ordering.
- Combined with a trained imaging/clinical model, this becomes the diagnostic analogue of AI-Calibration §9 — the expert ordering benchmarks or trains the model.
Validation
Prospective accuracy against confirmed final dx (top-1/top-3, AUC); head-to-head vs any incumbent diagnostic rule; calibration of the posterior probabilities; case-mix audit to confirm spectrum coverage; ongoing drift monitoring (dynamic BT). Pre-register the validation cohort and metrics. Report the QI-vs-research determination up front.
Common pitfalls
(a) Case-mix bias — the dominant failure; a bank missing the ambiguous presentations gives an ordering that's accurate where it doesn't matter and wrong where it does. (b) Circular validation — final-dx info leaking into the elicitation. (c) Eliciting base-rate prevalence instead of feature-conditional likelihood — define the construct precisely. (d) Treating the ordering as deployable before prospective validation + SaMD QMS. (e) Ignoring drift — disease epidemiology and practice shift; monitor with dynamic BT. (f) Confusing it with Classification §3 — Classification assigns one grade; Diagnostic orders competing conditions by conditional likelihood.
How it works — show me the method
Item selection. Items are (feature profile, candidate condition) combinations. Build a bank of realistic presentations spanning the differential, satisfying Strength-Spread §1.1 so the bank includes clear-cut presentations and the genuinely ambiguous ones where decision support actually helps. Case-mix bias is the central threat (an external-truth archetype): a convenience sample that misses the borderline presentations yields an ordering that looks accurate but fails exactly where it's needed. Anchor presentations pre-tested at Stage 0 Gate A.
What you compare. The elicited worth is diagnostic likelihood conditional on the feature profile, not base-rate prevalence and not severity. Define the feature set as the operationally-available work-up data (history, exam, labs, imaging). Features enter the model as covariates (§7), which is what makes the ordering feature-conditional rather than a fixed average ranking.
Comparison structure. Bare-vote, binary forced-choice pairwise "given these features, which diagnosis is more likely?", NO ties (ADR-CLIOH-03). BIBD pair allocation; ATRD Round 2 (TRE) on contested pairs (ADR-CLIOH-04). Covariate-structured BT (Springall 1973; Cattelan 2012) is the core of this archetype — worth(condition) = f(feature profile) — and is the §12-specific add-on (with EHR/dashboard delivery). Single-step back button (ADR-CLIOH-02). For a tool kept current as practice/epidemiology shifts, dynamic BT monitors drift.
Panel. Diagnosticians who actually work up these presentations — ED physicians, the relevant subspecialists; 15–30 (target 25–30; if N<15, Cooke-weighted aggregation). Recruiting the eventual users improves both face validity and adoption of any resulting widget.
External ground truth. External and required. Validate the feature-conditional ordering against the confirmed final diagnosis (path, culture, operative findings, definitive imaging, or adjudicated clinical follow-up). Primary metrics: top-1 / top-3 accuracy and ROC/AUC per condition. Beware circular validation (final-dx information leaking into the expert elicitation) and case-mix bias (validation cohort missing the spectrum). Keep the elicitation blinded to outcome.
Statistical backbone. Covariate-structured BT via choix (bootstrap CIs, Firth), with features as covariates → a feature-conditional worth for each condition, naturally read as a posterior over the differential. Bayesian-hierarchical BT is well-motivated here (the posterior-ordering interpretation, partial pooling across feature subgroups), not merely conditional (ADR-CLIOH-07 draft; Playbook §7). Validate against final dx; recalibrate on drift. The expert-elicited ordering can also serve as an informative prior for a subsequent data-driven diagnostic model (cf. Risk-Factor §7 prior-then-update logic).
For researchers — reporting, IRB & grant language
Reporting standards. TRIPOD / TRIPOD-AI (the ordering is a multinomial prediction model); STARD for the diagnostic-accuracy validation; CLAIM if an imaging-AI component is involved; DECIDE-AI for early clinical evaluation of any deployed widget. LitGuard every citation; DLRP Zotero mirror.
IRB / ethics. Human-subjects (patient-derived presentations + final-dx linkage); QI-vs-research determination up front. No PHI in the elicitation instrument; aggregate-only display; deployment-phase data governance is separate and heavier. The decision-support framing raises the bar: do not present an unvalidated ordering as a diagnostic aid, and any live deployment is SaMD with its own regulatory pathway. No unverified IRB claim in the UI.
Grant-application language.
"Differential-diagnosis ordering for high-stakes pediatric presentations — for example distinguishing septic arthritis from transient synovitis in the limping child, the territory of established clinical prediction rules — remains heavily reliant on tacit expert judgment that varies across providers. Diagnostic Hierarchy CLIOH elicits diagnosticians' pairwise judgments of conditional likelihood ('given these features, which diagnosis is more likely?') and fits a covariate-structured Bradley–Terry model in which each candidate condition's worth is a function of the presenting feature profile, yielding a feature-conditional posterior ordering of the differential. The ordering is validated against confirmed final diagnoses (top-1/top-3 accuracy and ROC/AUC) and, pending prospective validation and an SaMD-grade quality management system, can be delivered as an EHR decision-support widget — bringing measured, reproducible expert judgment to the point of care while keeping the validation and regulatory obligations explicit." (Verify the specific clinical-prediction-rule citation in the full LitGuard sweep before external use.)