AIC Seminar Series
Apprenticeship Learning via Inverse Reinforcement Learning
|Pieter Abbeel||Stanford University|
Date: Thursday, September 23rd 2004 at 4:00pm
Location: EJ291 (Directions)
In typical decision making and optimal
control problems, the goal is to find a policy that performs well under
the given dynamics and reward function. In this talk, I will first give a
brief introduction on Markov Decision Processes (MDP) and then
consider learning in an MDP where we are not explicitly given a reward
function, but where instead we can observe
an expert demonstrating the task that
we want to learn to perform. This setting
is useful in applications (such as the task of
driving) where it may be difficult to write
down an explicit reward function specifying
exactly how different desiderata should be
traded off. The expert can be seen as trying
to maximize a reward function that is expressible
as a linear combination of known
features, and I will give an algorithm for learning
the task demonstrated by the expert. The algorithm
is based on using inverse reinforcement
learning to try to recover the unknown
reward function. I will show that the proposed algorithm
terminates in a small number of iterations,
and that even though it may never recover
the experts reward function, the policy output
by the algorithm will attain performance
close to that of the expert, where here performance
is measured with respect to the experts
unknown reward function.
The work I am presenting is joint work with Andrew Ng.
Pieter Abbeel is a Ph.D. student in computer science at Stanford
University, working with professor Andrew Ng. His research interests
include reinforcement learning, machine learning and graphical models.
Please arrive at least 10 minutes early as you will need to sign in by
following instructions by the lobby phone at Building E. (or call Wilma
Lenz at 650 859 4904, or Vicenta at Lopez at 650 859 5750). SRI is
located at 333 Ravenswood Avenue in Menlo Park. Visitors may park in the
parking lots off Fourth Street. Detailed directions to SRI, as well as maps,
are available from the Visiting AIC web page.
There are two entrances to SRI International located on Ravenswood Ave.
Please check the Builing E entrance signage.
©2017 SRI International 333 Ravenswood Avenue, Menlo Park, CA 94025-3493