Archetype 3
Classification / Nosology CLIOH
turn a severity continuum into outcome-validated categorical grades.
- Evidence standard
- external
- What you compare
- Real cases (radiographs, images, vignettes, multimodal packets)
- Panel
- Disease experts, multi-site (15–30)
- Output
- 3–5-tier categorical system with explicit interval-scale boundaries
- Validation
- Outcome linkage (primary) + interobserver κ (secondary)
What it's for
A CLIOH study that takes real cases, derives a continuous BT severity worth from expert pairwise comparisons, then places cutpoints to create a discrete categorical classification validated against patient outcomes.
"How should we divide [condition] into severity grades that predict treatment response and outcome?"
When to use it
(a) Existing classifications have poor interobserver reliability (Schatzker, Tönnis, K–L); (b) treatment choice depends on severity; (c) you have a multi-site case bank with outcomes; (d) cutpoint placement is operationally consequential; (e) the field is willing to adopt a new system.
When not to
(a) Cases don't vary on a single latent severity dimension → Phenotype Boundary §11; (b) no outcome data (you'd be back to a Discovery/opinion exercise — §1); (c) variables not cases → §1; (d) the existing classification is already strongly outcome-correlated.
What you get
A 3–5-tier categorical classification with explicit interval-scaled boundaries; an interobserver-agreement study; an outcome-validation curve (event rate by grade).
A real example
- CLIOH-MSKI severity — ~100 pediatric MSKI cases, ~25 CORTICES surgeons, BT severity scaling, cutpoints validated against length-of-stay, ICU admission, and surgical-revision rate. (Today MSKI is Discovery §1 on severity drivers; the case-based version is this archetype.) Direct methodological precedent: the i-ROP VSS validation (PMID 35157950) — also the template for Hybrid H3 (Classification × AI-Calibration).
Validation
Primary: outcome linkage (do the grades predict the outcome better than the incumbent classification, by AUC?). Secondary: interobserver κ. Replication at an independent CORTICES site. Guard against circular validation (outcome data must NOT leak into the expert ratings) and case-mix bias.
Common pitfalls
(a) Convenience-sampled cases that miss the severity spectrum; (b) cutpoints chosen for statistical separation that have no clinical actionability; (c) circular validation (outcomes leaked into ratings); (d) ignoring case-mix bias; (e) reaching for ties to handle "these two look equally bad" — keep no-tie; the BT model handles near-ties via probability near 0.5.
How it works — show me the method
Item selection. Closed, spectrum-exhaustive: sample 60–150 real cases covering the full severity range (mild → severe). Convenience samples that miss the extremes are the cardinal failure. This is the case-based analogue of Strength-Spread §1.1 — the case set must span the dimension.
What you compare. Items are cases (images/radiographs/vignettes/multimodal packets), not variables. The latent dimension is severity; everything in the packet that informs severity is fair game.
Comparison structure. Bare-vote, binary forced-choice, no ties (ADR-CLIOH-03) on "which case is more severe?" BIBD allocation; ATRD Round 2 on contested pairs. (DR's "Adaptive CJ for efficiency" → future enhancement; BIBD is the default. With 60–150 cases, pair count is large — consider phased BIBD or r-reduction per Playbook §2, NOT ties.)
Panel. 15–30 disease experts across institutions (multi-site is important so cutpoints generalize). Stratify by site for subgroup/heterogeneity analysis.
External ground truth. Required. Each case linked to outcomes (e.g., MSKI: length-of-stay, ICU admission, surgical revision, recurrence). This is the defining property — without outcomes you cannot place or validate cutpoints.
Statistical backbone. Frequentist BT for case worths (choix, bootstrap CIs) as the primary scale. Then latent-class / finite-mixture models on the BT scale to identify natural breakpoints, and ROC-based outcome validation of the chosen cutpoints. Bayesian hierarchical warranted when sites are few / case-mix imbalanced (conditional path, ADR-CLIOH-07 draft). Precedent: Phelps et al. 2015 (PMID 25539230); Pedersen et al. 2021 (PMID 34006025).
For researchers — reporting, IRB & grant language
Reporting standards. STARD; COSMIN; CLAIM / TRIPOD-AI if AI-assisted. Pre-register on OSF. LitGuard + DLRP.
IRB / ethics. Human-subjects research. De-identified case bank under data-use agreement; CORTICES central IRB. No PHI in the instrument; aggregate-only public display. No unverified IRB claim in the UI (Hard Rule #7).
Grant-application language.
"Existing ordinal classifications of [condition] (e.g., Schatzker, Tönnis) were developed by unstructured committee consensus and exhibit poor interobserver reliability at exactly the grades where treatment choice diverges. Classification CLIOH derives a continuous Bradley–Terry severity scale from expert pairwise comparisons of real cases — an approach that has dominated ordinal grading in orthopaedic imaging (Pedersen et al. 2021, BMJ Open, AUC 0.97 vs 0.80 for Kellgren–Lawrence) and ophthalmology (i-ROP vascular severity scale, Ophthalmology 2022, PMID 35157950, correlation 0.90 with expert classification) — then places outcome-validated cutpoints to yield a reproducible categorical system."