See RAIL website for full list of ongoing projects and all publications.

RobotsFor.me

This project aims to develop a web-based, platform-independent tool for conducting large-scale robotics user studies online. Building upon the open source PR2 Remote Lab, RobotsFor.Me (http://RobotsFor.me) enables users to remotely control robots through a common web browser, enabling researchers to conduct user studies with participants across the globe. Still under development, the current version of RobotsFor.Me includes a web interface for robot control, as well as a Robot Management System for managing users and different study conditions. The complete system is platform independent, and to date has been tested with the Rovio, youBot and PR2 platforms in both physical and simulated environments. RobotsFor.Me is the first robot remote lab designed for use by the general public.

  • Russell Toris and Sonia Chernova. RobotsFor.Me and Robots For You. Interactive Machine Learning Workshop, Intelligent User Interfaces Conference, 2013.

Learning from Demonstration Comparison Study

Robot learning from demonstration (LfD) research focuses on algorithms that enable a robot to learn new task policies from demonstrations performed by a human teacher.  See the Survey of Robot Learning from Demonstration for more information on this research area.  This project contributed the first comparative evaluation of three leading algorithms in this area.

  • Halit Bener Suay, Russell Toris and Sonia Chernova.  A Practical Comparison of Three Robot Learning from Demonstration Algorithms.  International Journal of Social Robotics, special issue on Learning from Demonstration, Volume 4, Issue 4, Page 319-330, 2012.
  • Halit Bener Suay and Sonia Chernova. A Comparison of Two Algorithms for Robot Learning from Demonstration. In the IEEE International Conference on Systems, Man, and Cybernetics, 2011.
  • Halit Bener Suay and Sonia Chernova. Effect of the Human Guidance and State Space Size on Interactive Reinforcement Learning. In the IEEE International Symposium on Robot and Human Interactive Communication (Ro-Man), 2011.

CloudPrimer: Leveraging Common Sense Computing to Promote Early Childhood Literacy


Providing young children with opportunities to develop early literacy skills is important to their success in school, their success in learning to read, and their success in life. This project focuses on the creation of a new interactive reading primer technology on tablet computers that will foster early literacy skills and shared parent-child reading through the use of a targeted discussion-topic suggestion system aimed at the adult participant.  The Cloud Primer will crowdsource the interactions and discussions of parent-child dyads across a community of readers. It will then leverage this information in combination with a common sense knowledge base to develop computational models of the interactions. These models will then be used to provide context-sensitive discussion topic suggestions to parents during the shared reading activity with young children.

  • Adrian Boteanu and Sonia Chernova. Modeling Discussion Topics in Interactions with a Tablet Reading Primer. International Conference on Intelligent User Interfaces, 2013 (to appear).
  • Adrian Boteanu and Sonia Chernova. Modeling Topics in User Dialog for Interactive Tablet Media. Workshop on Human Computation in Digital Entertainment at the Eighth Annual AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, 2012.

Human-Agent Transfer

Human-Agent Transfer (HAT) is a policy learning technique that combines transfer learning, learning from demonstration and reinforcement learning to achieve rapid learning and high performance in complex domains.

  • Matthew E. Taylor and Sonia Chernova. Integrating Human Demonstration and Reinforcement Learning: Initial Results in Human-Agent Transfer. In the Proceedings of the Workshop on Agents Learning Interactively from Human Teachers at AAMAS 2010.
  • Matthew Taylor, Halit Bener Suay and Sonia Chernova. Integrating Reinforcement Learning with Human Demonstrations of Varying Ability. In the International Conference on Autonomous Agents and Multi-Agent Systems (AAMAS), Taipei, Taiwan, 2011.
  • Matthew E. Taylor, Halit Bener Suay and Sonia Chernova. Using Human Demonstrations to Improve Reinforcement Learning. In the AAAI 2011 Spring Symposium: Help Me Help You: Bridging the Gaps in Human-Agent Collaboration, Palo Alto, CA, 2011.

Crowdsourcing Human-Robot Interaction

The development of hand-crafted action and dialog generation models for a social robot is a time consuming process that yields a solution only for the relatively narrow range of interactions envisioned by the programmers. We are exploring a data-driven solution for interactive behavior generation that leverages online games as a means of collecting large-scale data corpora for human-robot interaction research.

  • Sonia Chernova, Jeff Orkin and Cynthia Breazeal. Crowdsourcing HRI through Online Multi-Player Games. In the AAAI Fall Symposium on Dialog with Robots, 2010.
  • Sonia Chernova and Cynthia Breazeal. Learning Temporal Plans from Observation of Human Collaborative Behavior. In the AAAI Spring Symposium: It’s All in the Timing: Representing and Reasoning About Time in Interactive Behavior, 2010.
  • Sonia Chernova, Nick DePalma, Cynthia Breazeal. Crowdsourcing Real World Human-Robot Dialog and Teamwork through Online Multiplayer Games. AI Magazine, Vol 32, No 4, 2011.
  • Cynthia Breazeal, Nick DePalma, Jeff Orkin, Sonia Chernova and Malte Jung. Crowdsourcing Human-Robot Interaction: New Methods and System Evaluation in a Public Environment. Journal of Human-Robot Interaction, (to appear)

Survey of Robot Learning from Demonstration

Learning from demonstration is an interactive technique in which a robot learns a policy based on example state to action mappings provided by a human teacher. A survey of robot learning from demonstration methods:

  • Brenna Argall, Sonia Chernova, Manuela Veloso and Brett Browning. A Survey of Robot Learning from Demonstration. Robotics and Autonomous Systems. Vol. 57, No. 5, pages 469-483, 2009.

Confidence-Based Autonomy

Confidence-Based Autonomy (CBA) is a mixed-initiative single robot demonstration learning algorithm that enables the robot and teacher to jointly control the learning process and selection of demonstration training data. This algorithm enables the robot to identify the need for and request demonstrations for specific parts of the state space based on confidence thresholds characterizing the uncertainty of the learned policy. The robot's demonstration requests are complemented by the teacher's ability to provide supplementary corrective demonstrations in error cases. Option classes enable choices between multiple equally applicable actions to be represented explicitly within the robot's policy.

  • Sonia Chernova and Manuela Veloso. Interactive Policy Learning through Confidence-Based Autonomy. Journal of Artificial Intelligence Research. Vol. 34, 2009.
  • Sonia Chernova and Manuela Veloso. Teaching Collaborative Multi-Robot Tasks through Demonstration. In IEEE-RAS International Conference on Humanoid Robots, Daejeon, Korea, December 2008.
  • Sonia Chernova and Manuela Veloso. Learning Equivalent Action Choices from Demonstration. In International Conference on Intelligent Robots and Systems (IROS 2008), Nice, France, September 2008.
  • Sonia Chernova and Manuela Veloso. Teaching Multi-Robot Coordination using Demonstration of Communication and State Sharing (Short Paper). Proc. of 7th Int. Conf. on Autonomous Agents and Multiagent Systems (AAMAS 2008), Padgham, Parkes, Muller and Parsons (eds.), May 12-16, 2008, Estoril, Portugal.
  • Sonia Chernova and Manuela Veloso. Multi-Thresholded Approach to Demonstration Selection for Interactive Robot Learning. In the 3rd ACM/IEEE International Conference on Human-Robot Interaction (HRI'08), March 12-15 2008, Amsterdam, The Netherlands.
  • Sonia Chernova and Manuela Veloso. Multiagent Collaborative Task Learning through Imitation. In the 4th International Symposium on Imitation in Animals and Artifacts, April 2007.
  • Sonia Chernova and Manuela Veloso. Confidence-Based Policy Learning from Demonstration Using Gaussian Mixture Models. In Proceedings of International Conference on Autonomous Agents and Multiagent Systems (AAMAS'07), May 2007.

Multi-Robot Learning from Demonstration

Based on the single-robot Confidence-Based Autonomy algorithm, we introduce a task and platform independent multi-robot demonstration learning framework that enables a single person to teach multiple robots. Building upon this framework, we formalize three approaches to teaching emergent collaborative behavior based on different information sharing strategies. We provide detailed evaluations of all algorithms in multiple simulated and robotic domains, and present a case study analysis of the scalability of the presented techniques using up to seven robots.

  • Sonia Chernova and Manuela Veloso. Confidence-Based Multi-Robot Learning from Demonstration. International Journal of Social Robotics, Volume 2, Number 2, pages 195-215, 2010.
  • Sonia Chernova and Manuela Veloso. Teaching Collaborative Multi-Robot Tasks through Demonstration. In IEEE-RAS International Conference on Humanoid Robots, Daejeon, Korea, December 2008.

Hands-Free Mobile Human-Robot Teaming

We demonstrate that structured light-based depth sensing with standard perception algorithms can enable mobile peer-to-peer interaction between humans and robots and present a robotic system that integrates person and gesture recognition, with speech recognition and synthesis, resulting in a natural and hands-free interactive system that requires little training.

  • Matthew Loper, Odest Chadwicke Jenkins, Nathan Koenig, Sonia Chernova, Chris Jones. Mobile Human-Robot Teaming with Environmental Tolerance. In the 4th ACM/IEEE International Conference on Human-Robot Interaction (HRI'09), March 9-13 2008, La Jolla, USA.
  • N. Koenig, S. Chernova, C. Jones, M. Loper, and O. Jenkins. Hands-free interaction for human-robot teams. In ICRA 2008 Workshop on Social interaction with intelligent indoor robots, Pasadena, CA, USA, May 2008.
  • N. Koenig, S. Chernova, C. Jones, M. Loper, and O. Jenkins. Hands-free human-robot interaction. In AAAI 2008 Video Program, Chicago, IL, USA, Jul 2008.
  • N. Koenig, S. Chernova, C. Jones, M. Loper, and O. Jenkins. Hands-free human-robot interaction. In HRI 2008 Video Program, Amsterdam, Netherlands, Mar 2008.

Cognitively-Inspired Action Selection

Long-term human–robot interaction, especially in the case of humanoid robots, requires an adaptable and varied behavior base. We present a method for capturing, or learning, sequential tasks by transferring serial behavior execution from deliberative to routine control. The incorporation of this approach leads to the natural development of complex and varied behaviors, with lower demands for planning, coordination and resources.

  • Sonia Chernova and Ronald C. Arkin. From Deliberative to Routine Behaviors: a Cognitively-inspired Action Selection Mechanism for Routine Behavior Capture. Adaptive Behavior Journal, Vol. 15, No. 2, pages 199-216, 2007.

Other Research Projects

  • Manuela Veloso, Nicholas Armstrong-Crews, Sonia Chernova, Elisabeth Crawford, Colin McMillen, Maayan Roth, Douglas Vail, and Stefan Zickler. A Team of Humanoid Game Commentators. International Journal of Humanoid Robotics. Vol. 5, No. 3, pages 457-480, 2008.
  • Sonia Chernova and Manuela Veloso. Tree-Based Policy Learning in Continuous Domains through Teaching by Demonstration. In Modeling Others from Observations: Papers from the AAAI Workshop, ed. Gal Kaminka, David Pynadath, and Christopher Geib, 24-31. Technical Report WS-06-13. American Association for Artificial Intelligence, Menlo Park, California, 2006.
  • Manuela Veloso, Nicholas Armstrong-Crews, Sonia Chernova, Elisabeth Crawford, Colin McMillen, Maayan Roth and Douglas Vail. A Team of Humanoid Game Commentators. In Proceedings of the International Conference on Humanoid Robots, Genova, Italy, 2006.
  • Sonia Chernova, Elisabeth Crawford and Manuela Veloso. Acquiring Observation Models through Reverse Plan Monitoring, In Proceedings of 12th Portuguese Conference on Artificial Intelligence, 2005.
  • Sonia Chernova and Manuela Veloso. An Evolutionary Approach To Gait Learning For Four-Legged Robots. In International Conference on Intelligent Robots and Systems (IROS'04), September 2004.
  • Sonia Chernova and Manuela Veloso. Learning and Using Models of Kicking Motions for Legged Robots. In Proceedings of International Conference on Robotics and Automation (ICRA'04), May 2004.
  • Manuela Veloso, Scott Lenser, Doug Vail, Paul Rybski, Nick Aiwazian, and Sonia Chernova. CMRoboBits: Creating an Intelligent AIBO Robot. In AAAI Spring Symposium on Accessible, Hands-on AI and Robotics Education, Palo Alto, CA, 2004.