Skip to content

DriveIRL: Driving in Real Life with Inverse Reinforcement Learning

Authors: Tung Phan-Minh, Forbes Howington, Ting-Sheng Chu, Sang Uk Lee, Momchil S. Tomov, Nanxiang Li, Caglayan Dicle, Samuel Findler, Francisco Suarez-Ruiz, Robert Beaudoin, Bo Yang, Sammy Omari, Eric M. Wolff

Published: 2022 (Conference Paper)

Source: IEEE International Conference on Robotics and Automation (ICRA)

Algorithm: DriveIRL

arXiv: 2206.03004

DOI: 10.1109/ICRA48891.2023.10160449

Summary

Inverse Reinforcement Learning-based planner demonstrated on a real self-driving car in dense urban traffic. Trained on large-scale human driving logs. The architecture design is critical to the success of the approach: there is a classical trajectory generator (based on Dubins paths, pre-computed acceleration profiles, and access to a clean road geometry model) capable of generating diverse safe trajectories, a safety filter that removes all trajectory candidates that are not forward recursively safe, and the learned model is only for assigning scores to the safety-filtered trajectory candidates. Includes a useful set of standardized evaluation metrics for learned planners (see Appendix A.5).

Abstract

In this paper, we introduce the first published planner to drive a car in dense, urban traffic using Inverse Reinforcement Learning (IRL). Our planner, DriveIRL, generates a diverse set of trajectory proposals and scores them with a learned model. The best trajectory is tracked by our self-driving vehicle's low-level controller. We train our trajectory scoring model on a 500+ hour real-world dataset of expert driving demonstrations in Las Vegas within the maximum entropy IRL framework. DriveIRL's benefits include: a simple design due to only learning the trajectory scoring function, a flexible and relatively interpretable feature engineering approach, and strong real-world performance. We validated DriveIRL on the Las Vegas Strip and demonstrated fully autonomous driving in heavy traffic, including scenarios involving cut-ins, abrupt braking by the lead vehicle, and hotel pickup/dropoff zones. Our dataset, a part of nuPlan, has been released to the public to help further research in this area.

Primary

Paper arxiv.org

Standard

arXiv Abstract 2206.03004 arXiv PDF 2206.03004 arXiv HTML 2206.03004 DOI 10.1109/ICRA48891.2023.10160449

Alternate

ieeexplore.ieee.org ieeexplore.ieee.org

Tags

  • Autonomous driving

  • Inverse reinforcement learning

  • Motion planning

  • Imitation learning

  • Real-world deployment