Authors:Anthony Francis (1), Claudia Pérez-D'Arpino (2), Chengshu Li (3), Fei Xia (4), Alexandre Alahi (5), Rachid Alami (15), Aniket Bera (6), Abhijat Biswas (7), Joydeep Biswas (8), Rohan Chandra (8), Hao-Tien Lewis Chiang (4), Michael Everett (10), Sehoon Ha (11), Justin Hart (8), Jonathan P. How (9), Haresh Karnan (8), Tsang-Wei Edward Lee (4), Luis J. Manso (12), Reuth Mirksy (13), Sören Pirk (14), Phani Teja Singamaneni (15), Peter Stone (8,16), Ada V. Taylor (7), Peter Trautman (17), Nathan Tsoi (18), Marynel Vázquez (18), Xuesu Xiao (19), Peng Xu (4), Naoki Yokoyama (11), Alexander Toshev (20), Roberto Martín-Martín (8) ((1) Logical Robotics, (2) NVIDIA, (3) Stanford, (4) Google, (5) EPFL, (6) Purdue, (7) CMU, (8) UT Austin, (9) MIT, (10) Northeastern, (11) Georgia Tech, (12) Aston, (13) Bar Ilan, (14) Adobe, (15) LAAS-CNRS, Universite de Toulouse, (16) Sony AI, (17) Honda, (18) Yale, (19) GMU, (20) Apple)
Abstract:A major challenge to deploying robots widely is navigation in human-populated environments, commonly referred to as social robot navigation. While the field of social navigation has advanced tremendously in recent years, the fair evaluation of algorithms that tackle social navigation remains hard because it involves not just robotic agents moving in static environments but also dynamic human agents and their perceptions of the appropriateness of robot behavior. In contrast, clear, repeatable, and accessible benchmarks have accelerated progress in fields like computer vision, natural language processing and traditional robot navigation by enabling researchers to fairly compare algorithms, revealing limitations of existing solutions and illuminating promising new directions. We believe the same approach can benefit social navigation. In this paper, we pave the road towards common, widely accessible, and repeatable benchmarking criteria to evaluate social robot navigation. Our contributions include (a) a definition of a socially navigating robot as one that respects the principles of safety, comfort, legibility, politeness, social competency, agent understanding, proactivity, and responsiveness to context, (b) guidelines for the use of metrics, development of scenarios, benchmarks, datasets, and simulators to evaluate social navigation, and (c) a design of a social navigation metrics framework to make it easier to compare results from different simulators, robots and datasets.
Submission history
From: Anthony Francis Jr [view email]
[v1]
Thu, 29 Jun 2023 07:31:43 UTC (8,053 KB)
[v2]
Mon, 14 Aug 2023 20:24:50 UTC (8,053 KB)
[v3]
Mon, 28 Aug 2023 23:19:14 UTC (8,054 KB)
[v4]
Tue, 19 Sep 2023 20:02:06 UTC (8,483 KB)