Rapid and accurate estimates of seismic ground failure and building damage are beneficial to efficient emergency response and post-earthquake recovery. Traditional approaches, such as physical and geospatial models, have poor accuracy and resolution due to large uncertainties and the limited availability of informing geospatial layers. The introduction of remote sensing techniques has shown potential in providing supplementary information for rapid hazard estimation by analyzing earthquake-induced correlation changes between pre- and post-event satellite images. However, the changes in satellite images are the result of overlapping ground failure, building damage, and environmental noise, making it challenging to categorize and estimate different seismic hazards and impacts directly from satellite images.Here we design a novel causality-informed Bayesian network that continuously updates seismic ground failure and building damage estimates from satellite images by modeling the physical interdependencies between geospatial features, ground failure, building footprints, building damage, and satellite images. The incorporation of physical interdependencies allows an effective fusion of physical models and rich but noisy information from remote sensing observations and reduces bias and uncertainties in estimations. Our experiments show that integrating satellite images through our Bayesian network improves the accuracy of seismic ground failure and building damage estimations.