As human impacts from wildfires mount, there is a pressing need to understand why structures are lost in destructive fires. Despite growing research on factors contributing to structure loss, fewer studies have focused on why some fires are destructive and others are not. We characterized overall differences between fires that resulted in structure loss (“destructive fires”) and those that did not (“non-destructive wildfires”) across three California regions. Then, we performed statistical analyses on large fires only (≥100 ha) to distinguish the primary differences between large destructive large fires and large non-destructive fires. Overall, destructive fires were at least an order of magnitude larger than non-destructive fires, with the largest area burned varying by season in different regions. Fire severity was also significantly higher in destructive than non-destructive fires. The statistical analysis showed that, in the San Francisco Bay Area and the northern Sierra Nevada foothills, proximity to the Wildland Urban Interface (WUI) was by far the most important factor differentiating destructive and non-destructive wildfires, followed by different combinations of short-term weather, seasonal climate, topography, and vegetation productivity. In Southern California, wind velocity on the day of the fire ignition was the top factor, which is consistent with previous assumptions that wind-driven fires tend to be most destructive and most of the destruction occurs within the first 24 h. Additionally, Southern California’s high population density increases the odds that a human-caused wildfire may occur during a severe fire-weather event. The geographical differences among regions and the variation of factors explaining the differences between large destructive and large non-destructive fires reflects the complexity inherent in decision-making for reducing wildfire risk. Land use planning to reduce future exposure of housing development to fire and increased focus on wildfire ignition prevention emerge as two approaches with substantial potential.