I explore why physics‐based models of earthquake triggering rarely outperform statistical models in prospective testing, outside of limited spatial‐temporal windows. Pseudo‐prospective tests on suites of synthetic aftershock sequences show that a major factor is the level of unmodeled spatial clustering of the direct aftershocks triggered by the mainshock. The synthetic sequences are generated from generalized “physical” triggering models, optionally superimposed on background heterogeneity that controls the level of clustering. The statistical Epidemic Type Aftershock Sequence (ETAS) model performs relatively better the more clustered the direct aftershocks, while the true generalized “physical” model performs relatively worse. Real aftershocks appear to be sufficiently clustered to allow ETAS to perform as well as or better than physical models such as Coulomb stress triggering. A likely cause of the spatial clustering of direct aftershocks is heterogeneity of the background physical conditions, which typically is not modeled in physics‐based forecasts. This implies that the forecast performance of physical models could be substantially improved through a better understanding of the interaction between earthquake stress changes and variable background physical conditions such as stress state, fault strength, and fluid pressure.