MARKOV: A methodology for the solution of infinite time horizon MARKOV decision processes
Applied Stochastic Models and Data Analysis
By: B. Kenneth Williams
Links
- More information: Publisher Index Page (via DOI)
- Document: Report (html)
- Download citation as: RIS | Dublin Core
Abstract
Algorithms are described for determining optimal policies for finite state, finite action, infinite discrete time horizon Markov decision processes. Both value-improvement and policy-improvement techniques are used in the algorithms. Computing procedures are also described. The algorithms are appropriate for processes that are either finite or infinite, deterministic or stochastic, discounted or undiscounted, in any meaningful combination of these features. Computing procedures are described in terms of initial data processing, bound improvements, process reduction, and testing and solution. Application of the methodology is illustrated with an example involving natural resource management. Management implications of certain hypothesized relationships between mallard survival and harvest rates are addressed by applying the optimality procedures to mallard population models.
Suggested Citation
Williams, B.K., 1988, MARKOV: A methodology for the solution of infinite time horizon MARKOV decision processes: Applied Stochastic Models and Data Analysis, v. 4, no. 4, p. 253-271, https://doi.org/10.1002/asm.3150040405.
| Publication type | Article |
|---|---|
| Publication Subtype | Journal Article |
| Title | MARKOV: A methodology for the solution of infinite time horizon MARKOV decision processes |
| Series title | Applied Stochastic Models and Data Analysis |
| DOI | 10.1002/asm.3150040405 |
| Volume | 4 |
| Issue | 4 |
| Publication Date | August 31, 2006 |
| Year Published | 1988 |
| Language | English |
| Contributing office(s) | Patuxent Wildlife Research Center |
| Description | 253-271 |
| Larger Work Type | Article |
| Larger Work Subtype | Journal Article |
| Larger Work Title | Applied Stochastic Models and Data Analysis |
| First page | 253 |
| Last page | 271 |