LLM Doesn't Know What It Doesn't Know: Detecting Epistemic Blind Spots via Cross-Model Attribution Divergence on Clinical Tabular Data

Researchers found that LLM-verbalized confidence scores are effectively meaningless on structured clinical data, outputting near-constant values regardless of whether the model's actual accuracy was 49% or 75%. By comparing attribution patterns between an LLM and XGBoost, they developed a cross-model calibration approach that reduced expected calibration error from 0.254 to 0.080 without accessing model internals or retraining. The findings are a measured but important caution for anyone deploying LLMs in healthcare settings where a model's stated certainty is treated as meaningful signal.