Plant communities in rangeland ecosystems vary widely in the degree to which they can compensate for losses to herbivores. Ecosystem-level factors have been proposed to affect this compensatory capacity, including timing and intensity of grazing, and availability of soil moisture and nutrients. Arid ecosystems are particularly challenging to predict because of their high degree of temporal variability in moisture inputs. We used a replicated herbivore exclusion experiment to evaluate herbaceous plant responses to grazing by large ungulates to test current theory and identify constraints on plant compensation in a dryland ecosystem. We measured nitrogen (N) yield and herbaceous production in three plant communities: meadows, willow-associated herbaceous communities, and riparian communities. We implemented grazing exclusion treatments from 2005 to 2008 in areas with elk and bison and areas with only elk. Grazing by large ungulates increased herbaceous production and N yield in herbaceous riparian communities. In willow communities, herbaceous plants displayed equal compensation in response to grazing in total aboveground production and N yield. Our results support the idea that plant compensation in this semiarid system is contingent on soil moisture availability, wherein the most productive sites (that received substantial moisture inputs from subsurface flow) exhibited overcompensation. Although the herbaceous riparian communities we studied are isolated patches of productive grassland in an otherwise shrub-dominated and minimally productive semiarid landscape, grazing by a combination of bison and elk removed only 44%–53% of aboveground net primary productivity (ANPP) during the growing season, and 25%–38% of production over winter. Consumption by ungulates was a positive linear function of herbaceous production, similar to reported patterns from other temperate and tropical grazing ecosystems. The slope of this relationship was affected by the analytical method used to calculate ANPP and consumption rates, but, regardless of the method, was lower or similar to reported slopes for other intensively grazed systems (Yellowstone, Serengeti, Laikipia) that have sustained high ungulate densities for decades to centuries. Given that the vegetation communities exhibited equal or overcompensation in terms of total herbaceous ANPP in both years, elk and bison population levels during our study period did not appear to occur at densities leading to degradation of herbaceous communities.