Actual correlations in n=1000
r=0: -0.0114 r=.33: 0.312 r=.5: 0.475
19 Patterns of External Validity
Fig 19.1: a missed predictor Z
Fig 19.2: a missed predictor Z with identical distribution
–> no impact
Fig 19.3: more or less severe cases selected
Fig 19.4: more or less heterogeneous cases selected
Fig 19.5: more or less severe cases selected by Z
–> Selection by missed predictor leads to miscalibration
Fig 19.6: more or less heterogeneous cases selected by Z
–> Selection by missed predictor has minor impact
Fig 19.7: Case-control design disturbs calibration
Disturbance is exactly as expected by ratio of selecting cases:controls (log(2)) Select all cases, and half of the controls –> same as shift in intercept
Fig 19.8: Overfitting disturbs discrimination and calibration
Fig 19.9: Different coeficients (model misspecification) disturbs discrimination and calibration
Scenarios: Table 19.4
Change of setting may especially impact calibration RCT vs survey may impact discrimination (more homogeneity) and calibration
