We develop multi-output neural network models (MNNs) to predict flow-duration curves (FDCs) in 9,203 ungaged locations in the Southeastern United States for six decades between 1950-2009. The model architecture contains multiple response variables in the output layer that correspond to individual quantiles along the FDC. During training, predictions are made for each quantile, and a combined loss function is used for back propagation and parameter updating. The loss function accounts for the covariance between the quantiles and generates physically consistent outputs (i.e., monotonically increasing quantiles with increasing nonexceedance probabilities). We use neural-network dropout to generate posterior-predictive distributions for FDCs, and test model performance under cross validation. Finally, we demonstrate how local surrgotate models, via the Local Interpretable Model-agnostic Explanations (LIME) method, can be used to infer the relation between basin characteristics and the predicted FDCs. Results suggest that MNNs can learn the monotonic relations between adjacent quantiles on an FDC, they result in better predictions than single output neural-network models that predict each quantile independently, and basin characteristics are most useful for predicting smaller quantiles, whereas bias terms from neighboring quantiles are most informative for predicting higher quantiles.