A citizen’s right to comment on, and criticize, government decisions makes a difference. The U.S. National Environmental Policy Act of 1969 (NEPA) institutionalized public engagement in environmental review in the belief it would lead to better decisions and more sustainable outcomes. But, 50 years later, NEPA’s public comment process has been criticized as costly and slow, while doing little to change outcomes. Data science now makes it possible to track progress and evaluate the influence of public participation. We examined 108 environmental impact statement (EIS) processes spanning 22 years. Our analysis revealed that public comments resulted in substantive decision alterations in 62% of cases, with 64% showing modifications to alternatives, 42% showing modifications to mitigation plans and 11% leading to the selection of an entirely new preferred alternative. When federal agencies changed project alternatives (78 EISs), 88% of the time (69 of the 78 EISs) they credited public comments as the reason. In 45 of the 108 EISs, agencies modified mitigation plans and credited public comments as the reason 100% of the time. Agencies only occasionally selected a new preferred alternative (21 out of 104 EISs), but when they did, they credited public comments as the reason 100% of the time. As the United States and the 190+ states and countries that have adopted NEPA’s example consider how to address environmental change, it is important to assess the role of public participation in environmental decision making. Our data say public comments matter.