19 Patterns of External Validity

Fig 19.1: a missed predictor Z

## Actual correlations in n=1000
## r=0: -0.0114  r=.33: 0.312  r=.5: 0.475

Fig 19.2: a missed predictor Z with identical distribution

–> no impact

Fig 19.3: more or less severe cases selected

Fig 19.4: more or less heterogeneous cases selected

Fig 19.5: more or less severe cases selected by Z

–> Selection by missed predictor leads to miscalibration

Fig 19.6: more or less heterogeneous cases selected by Z

–> Selection by missed predictor has minor impact

Fig 19.7: Case-control design disturbs calibration

Disturbance is exactly as expected by ratio of selecting cases:controls (log(2)) Select all cases, and half of the controls –> same as shift in intercept

Fig 19.8: Overfitting disturbs discrimination and calibration

Fig 19.9: Different coeficients (model misspecification) disturbs discrimination and calibration

Scenarios: Table 19.4

Change of setting may especially impact calibration RCT vs survey may impact discrimination (more homogeneity) and calibration

Uncertainty in validation simulations