Plot-scale experimentation can provide valuable insight into the effects of manure management practices on phosphorus (P) runoff, but whole-farm evaluation is needed for complete assessment of potential trade offs. Artificially-applied rainfall experimentation on small field plots and event-based and long-term simulation modeling were used to compare P loss in runoff related to two dairy manure application methods (surface application with and without incorporation by tillage) on contrasting Pennsylvania soils previously under no-till management. Results of single-event rainfall experiments indicated that average dissolved reactive P losses in runoff from manured plots decreased by up to 90% with manure incorporation while total P losses did not change significantly. Longer-term whole farm simulation modeling indicated that average dissolved reactive P losses would decrease by 8% with manure incorporation while total P losses would increase by 77% due to greater erosion from fields previously under no-till. Differences in the two methods of inference point to the need for caution in extrapolating research findings. Single-event rainfall experiments conducted shortly after manure application simulate incidental transfers of dissolved P in manure to runoff, resulting in greater losses of dissolved reactive P. However, the transfer of dissolved P in applied manure diminishes with time. Over the annual time frame simulated by whole farm modeling, erosion processes become more important to runoff P losses. Results of this study highlight the need to consider the potential for increased erosion and total P losses caused by soil disturbance during incorporation. This study emphasizes the ability of modeling to estimate management practice effectiveness at the larger scales when experimental data is not available.