Archetype 1

Research / Discovery CLIOH

rank candidate drivers of a phenomenon to aim the next study.

InternalLiveSCORE-AFCORTICES-Distal-TibiaOR-Priority

Evidence standard: internal
What you compare: Variables / risk / causal criteria
Panel: Domain experts, broad (15–30)
Output: Interval-scaled ranked variable list with 95% CIs
Validation: Convergence + test-retest (NOT outcome linkage)

What it's for

A CLIOH study that asks expert clinicians to make pairwise comparisons among candidate causal/risk variables for a phenomenon, producing a ranked, interval-scaled list of "what experts collectively believe drives this outcome," to direct subsequent empirical research.

"Of the dozens of plausible drivers of [arthrofibrosis / growth arrest / MSKI severity / OR urgency], which deserve to be measured in our next prospective study?"

When to use it

(a) A phenomenon is clinically important but mechanistically murky; (b) prior empirical work is sparse, conflicting, or absent; (c) a multicenter prospective dataset is being designed and variable selection is the bottleneck; (d) the team wants to operationalize tacit surgical intuition rather than chase the literature's stale variable list; (e) hypothesis generation — not validation — is the goal.

When not to

(a) You already have a large clean outcome dataset → go straight to regularized regression / SHAP; (b) you need a categorical grading system → Classification §3; (c) you need a real-time decision tool → Triage §4; (d) you want to certify learners → Calibration §5; (e) the variables are obvious and uncontested.

What you get

Caterpillar plot of variables ranked by worth with 95% CIs; ranked CSV; per-respondent divergence; free-text dissent appendix from ATRD. Unanimity Gate governs blinded vs unblinded respondent labels (ADR-CLIOH-05).

A real example

SCORE-AF (arthrofibrosis risk factors) — the founding Discovery deployment; k=19. The "female-sex paradox" (rank 11, 45% win rate) is only interpretable because the slate happened to span the strength range (the "by-accident" Strength-Spread case that motivated §1.1).
CORTICES-Distal Tibia (growth-arrest predictors) — Discovery with Risk-Factor flavor; flagship live instrument; the §1.1 audit flagged it as the all-contender / no-anchor cautionary case.
OR-Priority / ER-Priority / Interurban-2026 — Discovery of triage criteria (which factors drive urgency) — a precursor that becomes Triage §4 once deployed on real cases.
MSKI — Discovery of severity drivers; its outcome-severity dimension self-spans the range (the one "by-accident compliant" CORTICES instrument).
Methodological precedent: Pedersen et al. 2021 BMJ Open (PMID 34006025) — BT-derived severity dominated ordinal grading (AUC 0.97 vs 0.80 K–L).

Validation

(a) Test–retest at ~3 months on a subsample (embedded duplicates intra-session); (b) panel-composition sensitivity (drop-one-out); (c) convergent validity against later empirical regression once a CORTICES outcome dataset exists. Post-hoc Strength-Spread diagnostics (BT range ≥ 2.0 log-odds; anchor floor; contender CI separation).

Common pitfalls

(a) Over-pruning the seed list before elicitation; (b) all-contender slate / no anchors → compressed BT, overlapping CIs, unrankable (the DT failure mode — fix with §1.1); (c) treating internal consensus as truth without later empirical validation; (d) too narrow a panel (groupthink); (e) ignoring intransitivity — always run the transitivity check.

Maturation path

Discovery → (add real cases + outcome linkage) → Classification §3 or Triage §4 (Hybrids H3/H4). Discovery is correctly the first machine to build when the science is murky.

How it works — show me the method

Item selection. Open and inclusive. Seed list from literature review + free-text faculty nomination; aim for 20–60 candidates, trimmed to a deployable k = 13–19 (Playbook §1). Trim only obvious duplicates — do not pre-prune for face validity (that is the BT model's job). But you MUST satisfy Strength-Spread §1.1 (next field).

What you compare. Mix mechanistic, anatomic, demographic, procedural, and biomarker variables — heterogeneity is a feature. Strength-Spread is mandatory and binds hardest here: Discovery dimensions (etiology / risk-factor strength) usually have no natural floor/ceiling, so anchors must be designed deliberately at Stage 0 Gate A — HIGH contenders + MEDIUM bridges + LOW anchors (items the panel reliably ranks bottom, to calibrate the interval scale). Run the 3–5 rater anchor pre-test before deployment (Playbook §1.1).

Comparison structure. Bare-vote, binary forced-choice, NO ties (ADR-CLIOH-03). BIBD pair allocation (r=8 appearances/factor; Playbook §2). Within-pair position + pair order randomized (seeded PRNG). Round 2 = ATRD/TRE on contested pairs. (The DR's "Davidson-extended ties" is rejected for our default; revisiting ties would need a Mode-B ADR.)

Panel. 15–40 fellowship-trained clinicians, geographic + practice-volume diversity, ideally CORTICES sites. Minimum N=15, target 25–30 (Playbook §1). Stratify by site + years-in-practice for subgroup analysis.

External ground truth. None. The panel consensus is the output. This is the defining property of the archetype — be explicit in the paper that the claim is descriptive, not predictive.

Statistical backbone. Frequentist BT MLE via choix (Python), bootstrap 95% CIs, Firth penalty for separation (Playbook §5). Bayesian hierarchical BT (bpcs/PyMC) only as the subgroup / small-N path (conditional, per ADR-CLIOH-07 draft) — not the Stan-default the DR assumes. Report posterior/point worth + 95% CI per variable + between-rater heterogeneity.

For researchers — reporting, IRB & grant language

Reporting standards. CHERRIES (online survey), STROBE-style flow, BayesWatch if Bayesian. Pre-register on OSF with locked seed-list + SAP. LitGuard every citation; DLRP Zotero mirror.

IRB / ethics. Typically non-human-subjects or expedited (45 CFR 46.104(d)(2) — professional-opinion survey). No patient data. No IRB/IACUC claim embedded in the instrument UI unless verified (Hard Rule #7). Implied-consent Research Information Sheet; two-database identity separation.

Grant-application language.

"Despite [phenomenon X]'s clinical importance, mechanistic understanding is fragmented across small case series. We will use the Clinical Intuition Ordinal Hierarchy (CLIOH) framework — a structured pairwise-comparison protocol grounded in Bradley–Terry methodology now standard in educational assessment (Pollitt 2004; Verhavert et al. 2019) and AI preference learning, where reward models are fit with the identical BT likelihood (Christiano et al. 2017) — to elicit and quantify CORTICES clinicians' collective causal intuitions about [X], producing an interval-scaled, credibly-bounded variable hierarchy to power prospective study design."