Precisely Wrong: Conformal Prediction for Honest Precision Oncology
Date:
Stacy, C. L. and Das, S.
A selective-prediction system uses a model’s confidence score to decide when it should abstain, assuming that score reflects whether the prediction is correct. This poster examines a setting where it does not: a DNA methylation tumour-type classifier evaluated on TCGA across genetic-ancestry groups. Confident errors concentrate in ancestry groups under-represented in training, and deep-ensemble uncertainty gives no warning, because the errors come from bias rather than variance. Group-conditional conformal prediction gives prediction sets with valid coverage within each ancestry group, offering a third option between confidently predicting for patients unlike the training data and excluding them from precision medicine altogether.