Active Uncertainty Learning for HRI: An Implicit Dual Control Approach

Predictive models are effective in reasoning about human motion, a crucial part that affects safety and efficiency in human-robot interaction. However, robots often lack access to certain key parameters of such models, for example, human’s objectives, their level of distraction, and willingness to cooperate.

In order to plan safely and efficiently, the robot must effectively reason about the human’s future motion and their reactions to the robot’s actions. Also, the robot needs to identify the human’s hidden states that are not observable by the robot but are important to the planning quality. Those hidden states might include the human’s objective, whether they are cooperative, or the role they prefer to play when interacting with the robot.

Dual control theory addresses this challenge by treating unknown parameters as stochastic hidden states and identifying their values using information gathered during control of the robot. Despite its ability to optimally and automatically trade off exploration and exploitation, dual control is computationally intractable for general human-in-the-loop motion planning, mainly due to nested trajectory optimization and human intent prediction.

An example scenario tree that produces a dual control policy.

In this paper, we present a novel algorithmic approach to enable active uncertainty learning for human-in-the-loop motion planning based on the implicit dual control paradigm. Our approach relies on sampling-based approximation of stochastic dynamic programming, leading to a model predictive control problem that can be readily solved by real-time gradient-based optimization methods.

Our proposed ID-SMPC planner yielded a clean and sharp overtaking maneuver of the robot while the non-dual planners led to unsafe trajectories, due to lack of dual control effect. The ED-SMPC planner, even with fine tuning, was more conservative and unable to produce an efficient trajectory.

The resulting policy is shown to preserve the dual control effect for generic human predictive models with both continuous and categorical uncertainty. The efficacy of our approach is demonstrated with simulated driving examples.

Simulation snapshots of a multi-agent interaction scenario with a pedestrian and two human-driven vehicles using the ID-SMPC planner.
%d bloggers like this: