Small, surface-release dams are ubiquitous features of the landscape that typically slow water flow and decrease canopy cover through impounded reaches, potentially increasing stream temperatures. However, reported effects of small dams on water temperature are variable, likely due to differences in landscape and dam characteristics. To quantify the range of thermal effects of small dams, we deployed continuous temperature loggers for one to four years at 30 dam sites across a range of environmental settings throughout Massachusetts (USA). Most dams (67%) warmed downstream waters, with August mean temperatures 0.20–5.25 °C higher than upstream. Downstream temperatures cooled with increased distance from the dam at 68% of sites, such that the warmest temperatures were observed closest to the dam. Where there was both a significant downstream warming effect and cooling pattern (seven sites), elevated temperatures persisted for an average of 1.31 km downstream of the dam. Dams with impoundments that caused the greatest relative widening of the stream channel and those on coldwater streams had the most warming, while streams with short dams in forested watersheds cooled most quickly downstream of the dam. Flow had a homogenizing effect on water temperatures at over half of the sites, whereby summer thermal impacts were more pronounced (e.g., more warming, faster cooling rates) under periods of lower flows. Downstream warming may reduce habitat for coldwater fishes and invertebrates, particularly where dams shift coldwater/coolwater habitat to warmwater. These results suggest that dam removal may mitigate elevated stream temperatures and increase ecosystem resilience in the face of a changing climate via restoration of critical coldwater habitats.