Basic data on river bathymetry is critical for numerous applications in river research and management and is increasingly obtained via remote sensing, but the noisy, pixelated appearance of image‐derived depth maps can compromise subsequent analyses. We hypothesized that this noise originates from reflectance from an irregular water surface and introduced a framework for mitigating these effects by Inferring Bathymetry from Averaged River Images (IBARI). This workflow produces time‐averaged images from video frames stabilized to account for platform motion and/or computes a spatial average from an ensemble simulated by randomly shifting images relative to themselves. We used field observations of water depth and helicopter‐based videos from a clear‐flowing river to assess the potential of this approach to improve depth retrieval. Our results indicated that depths inferred from averaged images were more accurate and precise than those inferred from single frames; observed versus predicted regression R² increased from 0.80 to 0.88. In addition, IBARI significantly enhanced the texture of image‐derived depth maps, leading to smoother, more coherent representations of channel morphology. Depth retrieval improved with image sequence duration, but the number of images was more important than the length of time encompassed; shorter acquisitions at higher frame rates would economize data collection. We also demonstrated the potential to scale up the IBARI workflow by producing a mosaic of bathymetric maps derived from averaged images acquired at several hovering waypoints distributed along a 2.36 km reach. This approach is well‐suited to data collected from helicopters and small unmanned aircraft systems.