Autonomy Incubator Seminar Series:
TRACTABLE BAYESIAN NONPARAMETRIC MODELS (BNPs) FOR PLANNING UNDER UNCERTAINTY
Prof. Jonathan P. How, Dept. of Aeronautics and Astronautics, Massachusetts Institute of Technology
March 17, 2014, 10:00 am, NASA Langley, Reid Center
Abstract:
This talk will present recent results on model-based and model-free algorithms for planning under uncertainty for single and multiagent systems in complex, poorly known, and dynamic environments. The techniques developed could be used to enable unmanned ground vehicles to navigate through pedestrian and vehicle traffic and to land unmanned aerial vehicles at an uncontrolled airport. For the model-based approaches, Bayesian nonparametric models (BNPs) uniquely provide the flexibility to learn model size and parameters, which is important because it is often very difficult to pre-specify these parameters prior to the missions. We utilize Gaussian Processes to represent the trajectory velocity fields of the obstacles (static & dynamic) in the environment; a Dirichlet process GP mixture (DP-GP), which can learn the number of motion models and their velocity fields; and the dependent Dirichlet process GP mixture (DDP-GP), which captures the same quantities and temporal evolution. Robust planning is achieved by embedding these probabilistic models into a chance-constrained RRT* planner, and applications for urban driving will be presented. To reduce the computational effort in modeling, we also present the Dynamic Means algorithm, which is shown to be 1000 times faster than Gibbs sampling on a large data set.
The model-free approaches to continuous-state RL use GPs to model current estimate of the optimal value function Q* with off-policy updates. The key result is that we prove that convergence of the off-policy updates for the GP weights by properly setting a regularization-like GP hyperparameter. It is shown that GPQ reaches comparable performance (often faster) while choosing its own basis functions and thus requires less a priori information than other algorithms in the literature.
These algorithms address key issues in planning under uncertainty and also highlight the key role of planning to reduce uncertainty. Future work will investigate techniques to extend beyond DPs and GPs to consider the many other BNP models that have recently been developed.
See http://acl.mit.edu for further details and papers.
Biography:
Dr. Jonathan P. How is the Richard C. Maclaurin Professor of Aeronautics and Astronautics at the Massachusetts Institute of Technology. He received a B.A.Sc. from the University of Toronto in 1987 and his S.M. and Ph.D. in Aeronautics and Astronautics from MIT in 1990 and 1993, respectively. He then studied for two years at MIT as a postdoctoral associate for the Middeck Active Control Experiment (MACE) that flew on-board the Space Shuttle Endeavour in March 1995. Prior to joining MIT in 2000, he was an Assistant Professor in the Department of Aeronautics and Astronautics at Stanford University. He has graduated 36 Ph.D. students while at MIT and Stanford University on topics related to GPS navigation, multi-vehicle planning, and robust/hybrid control. He has published more than 295 articles in peer-reviewed proceedings and 85 papers in technical journals. Current research interests include: (1) Design and implementation of distributed robust planning algorithms to coordinate multiple autonomous vehicles in dynamic uncertain environments; (2) Reinforcement learning for real-time aerospace applications; and (3) Adaptive flight control to enable autonomous agile flight and aerobatics. Professor How was the planning and control lead for the MIT DARPA Urban Challenge team that placed fourth in the 2007 race at Victorville, CA. He was the recipient of the 2002 Institute of Navigation Burka Award, a Boeing Special Invention award in 2008, the 2011 IFAC Automatica award for best applications paper, Recipient of the AIAA Best Paper Award from the 2011 and 2012 Guidance Navigation and Control Conferences, and he is an Associate Fellow of AIAA and a senior member of IEEE.