Adaptive plasticity in thermal tolerance traits may buffer organisms against changing temperatures, making such responses of particular interest in the face of global climate change. Although population variation is integral to the evolvability of this trait, many studies inferring proxies of physiological vulnerability from thermal tolerance traits extrapolate data from one or a few populations to represent the species. Estimates of physiological vulnerability can be further complicated by methodological effects associated with experimental design. We evaluated how populations varied in their acclimation capacity (i.e., the magnitude of plasticity) for critical thermal maximum (CTmax) in two species of tailed frogs (Ascaphidae), cold-stream specialists. We used the estimates of acclimation capacity to infer physiological vulnerability to future warming. We performed CTmax experiments on tadpoles from 14 populations using a fully factorial experimental design of two holding temperatures (8 and 15°C) and two experimental starting temperatures (8 and 15°C). This design allowed us to investigate the acute effects of transferring organisms from one holding temperature to a different experimental starting temperature, as well as fully acclimated responses by using the same holding and starting temperature. We found that most populations exhibited beneficial acclimation, where CTmax was higher in tadpoles held at a warmer temperature, but populations varied markedly in the magnitude of the response and the inferred physiological vulnerability to future warming. We also found that the response of transferring organisms to different starting temperatures varied substantially among populations, although accounting for acute effects did not greatly alter estimates of physiological vulnerability at the species level or for most populations. These results underscore the importance of sampling widely among populations when inferring physiological vulnerability, as population variation in acclimation capacity and thermal sensitivity may be critical when assessing vulnerability to future warming.