Printer-friendly versionPDF version

Finance ANR | 2014 - 2018

Project leader: humbert Fiorino

SOMBRERO proposes to train humanoid robots (Nina and Roméo) i.e. provide them with socio-communicative abilities by immersive teleoperation i.e. “beaming” of human pilots. The SOMBRERO project has three main objectives:

  1. Realize beaming experiments involving adults interacting with a humanoid robot in cooperative, situated and finalized tasks. The targeted task is a series of tests used in neuropsychological records. We focus on the socio-communicative behavior that should accompany the monitoring of the task, i.e. the correct comprehension of the instructions, the seamless execution of the task by the interlocutor, the positive feedback and encouragement given along the performance, correction of errors, misunderstandings or precisions,
  2. Develop and implement autonomous socio-communicative behaviors into the robot cognitive architecture via statistical modeling of the multimodal behaviors monitored during the prior robot- mediated interactions,
  3. Assess these behaviors and the achieved social embodiment with acceptance measures and analysis of user attitudes. 

Figure 1. The SOMBRERO roadmap: robots acquire socio-communicative multimodal behaviors via a two-step procedure: (1- Beamed behaviors) During an observation phase, a behavioral memory collects on-line multimodal interaction traces of robot-mediated interactions between the pilot and its interlocutor. The pilot experiences “from the inside”1 the robotic embodiment by a direct coupling of the robot actuators and sensors with the pilot’s senses thanks to motion capture and audiovisual immersion. (2 – Autonomous behaviors) Then, a second, mainly off-line, process links the monitored multimodal traces with an a priori model of the mental processes involved in the scene analysis, comprehension and action generation. Within SOMBRERO, the raw multimodal scores will be semi-automatically enriched with annotations, tracing the mental processes and unveiling of decisions involved in the on-going evaluation of the situation.