Analysis of genetic connectivity helps to define stock boundaries and provides information on interpopulation dynamics, such as migration and spawning site fidelity. We used 16 microsatellite loci to describe the genetic population structure of 1,215 Smallmouth Bass Micropterus dolomieu from 32 sites throughout the upper Mississippi River and Lake Michigan watersheds. We found that Smallmouth Bass populations formed two genetically distinct units separated by the Mississippi River–Lake Michigan drainage boundary. Smallmouth Bass from the Lake Michigan drainage could be parsimoniously grouped into two or six genetically distinct units that largely corresponded with either river or lake habitats, while fish from the Mississippi River drainage grouped into two, six, or nine genetic units that were mostly associated with watershed boundaries. In the Lake Michigan and Mississippi River drainages, relative migration was limited between lake and river sites, suggesting that gene flow between neighboring sites with different habitat attributes can be low. Our research provides a higher‐resolution assessment of Smallmouth Bass genetic structure in a core portion of the species’ range and provides strong evidence that Smallmouth Bass populations are structured at small spatial scales that are potentially associated with habitat type. These results demonstrate the importance of evaluating genetic structure at small spatial scales and adopting management strategies that preserve genetic diversity of black bass populations at both the watershed level and the habitat level.