Proceedings

Forward Search Value Iteration for POMDPs

Download BibTeX Cite

Proceedings of the Twentieth International Joint Conference on Artificial Intelligence. Pages 2619-2623.

© 2007 International Joint Conferences on Artificial Intelligence

Abstract

Recent scaling up of POMDP solvers towards realistic applications is largely due to point-based methods which quickly converge to an approximate solution for medium-sized problems. Of this family HSVI, which uses trial-based asynchronous value iteration, can handle the largest domains. In this paper we suggest a new algorithm, FSVI, that uses the underlying MDP to traverse the belief space towards rewards, finding sequences of useful backups, and show how it scales up better than HSVI on larger benchmarks.

  1. [Barto et al., 1995] A. G. Barto, S. J. Bradtke, and S. P. Singh. Learning to act using real-time dynamic program- ming. Journal of AI, 72(1):81-138, 1995.
  2. [Bonet and Gefner, 1998] B. Bonet and H. Gefner. Solving large POMDPs using real time dynamic programming. In AAAI Fall Symposium on POMDPs, 1998.
  3. [Littman et al., 1995] M. L. Littman, A. R. Cassandra, and L. P. Kaelbling. Learning policies for partially observable environments: Scaling up. In ICML'95, 1995.
  4. [Pineau et al., 2003] J. Pineau, G. Gordon, and S. Thrun. Point-based value iteration: An anytime algorithm for POMDPs. In IJCAI, August 2003. IJCAI
  5. [Poupart and Boutilier, 2003] P. Poupart and C. Boutilier. Bounded finite state controllers. In NIPS 16, 2003.
  6. [Smallwood and Sondik, 1973] R. Smallwood and E. Sondik. The optimal control of partially observ- able processes over a finite horizon. OR, 21, 1973.
  7. [Smith and Simmons, 2005] T. Smith and R. Simmons. Point-based pomdp algorithms: Improved analysis and im- plementation. In UAI 2005, 2005.
  8. [Spaan and Vlassis, 2005] M. T. J. Spaan and N. Vlassis. Perseus: Randomized point-based value iteration for POMDPs. JAIR, 24:195-220, 2005.
  9. [Zhou and Hansen, 2001] R. Zhou and E. A. Hansen. An im- proved grid-based approximation algorithm for POMDPs. In IJCAI, pages 707-716, 2001. IJCAI

References were extracted automatically from the PDF and may contain errors.

Welcome Future Conferences AI Journal Division IJCAI Proceedings About IJCAI IJCAI Awards