Application: MiRA
1. OverviewMixed Reality Agents are embodied in both the virtual and the physical space. These agents combine the physical presence of a robot with the adaptability and expressitivity of a virtual character. The objective is to blur the traditional boundaries between the real and the virtual and bridge the gap between Human-Computer Interaction (HCI) and Human-Robot Interaction (HRI). To facilitate the development of and experimentation with mixed reality agents, we created the Socially Situated Agent Architecture (SoSAA). It investigates explicit deliberative control of socially and physically situated agents in virtual, real and mixed reality environments. The architecture embraces the fusion between deliberative social reasoning mechanisms and explicit tangible behavioural mechanisms for human-agent social interaction. 2. Introduction
However, profound technical and methodological differences exist when dealing with robotic and software domains. Nevertheless there is an emerging recognition that the construction of truly social agents could benefit from the accommodation of both schools of thought. Specifically, the popularity of social agents has also opened the question of how heterogeneous agent societies, e.g. communities of agents (hardware or software), can be integrated in order to perform useful tasks in human societies. We developed a novel integration methodology, which combines a physical robotic body and a virtual character displayed through a mixed reality overlay. We believe that such a construction, which we call MiRA (Mixed Reality Agent), offers a viable alternative to merging the offering of physical and virtual agents into one socially competent agent. 3. Prototyping Robotic Interfaces with MiRAs
Instead of confining the virtual character to a screen, we advocate the use of mixed reality visualisation via a Head-Mounted Display (HMD). This not only makes the virtual character visible from all angles but it offers other very compelling possibilities that a screen-based visualisation cannot fulfill. Since the character is rendered on each user's personal display, the virtual avatar is now only constrained by the user's field of view. As the agent is embodied in the real and the virtual world it can manipulate both and can interact with physical as well as with virtual objects. This interaction transcends the boundaries between the real and the virtual world; the virtual avatar can, for example, point at real things, while the physical robot can steer around virtual obstacles. As the virtual part is rendered individually on each user's equipment, it offers the compelling possibility of personalising the content for each user. A distinct disadvantage, at the moment, is the cumbersome and expensive hardware imposed on each user. However, this is likely to change in the near future as displays become cheaper and less invasive. 4. Ongoing ResearchEvaluating interaction with MiRAs is a difficult challenge, especially considering the added novelty factor caused by the use of mixed reality and the limitations of the current equipment. Currently, we are therefore most interested to prove the feasibility of our approach, to demonstrate the capability of the system, and to gather users' expectations and reactions in the scope of informing future developments, both in the architecture and in the evaluation methodology. In particular, we were interested in the following questions:
In future experiments, we hope to investigate the specifics of mixed reality agents further. In particular:
To answer these questions, we have planned a series of experiments to see where MiRAs exhibit different properties than purely virtual or purely physical agents. A pilot study has already shown that people ascribe more capabilities to a mixed reality agent than to a robot. By studying how the virtual agent's actions influence a person's mental model of the robot, we can examine if people perceive a MiRA as one holistic agent. Another experiment then will assess how a MiRA is perceived differently to a robotic or a virtual agent by the approach distances people feel comfortable with. 5. Developer VideosThese videos demonstrate how mixed reality can help in the development not only of the robotic interface but also to augment the robot and its environment in ways that are beyond a purely physical robot. All videos are available in MPEG format in a 640x480 resolution.
6. Pilot Study VideosWe have recently conducted an experiment to gather user reactions to our mixed reality agent (MiRA), results of which can be found in our paper for RO-MAN 2007 entitled Using Mixed Reality Agents as Social Interfaces for Robots. Below are some of the videos taken from that experiment. All videos are available in MPEG format in a 720x480 resolution.
7. Selected Publications
|
Artificial agents, be they of a robotic or purely software genre, are gradually populating the human social space. The pervasiveness of robotic platforms, in the form of entertainment robots, household appliances, or assisitive technologies, has acted as a catalyst for a large body of research exploring human-robot interaction. The ultimate goal of social robotics is to develop robots that socially interact with humans, working alongside people and acting as socially competent peers rather than mere tools depending on human supervision. Crucially, the same objectives are addressed in software-only domains, for example, in supporting the development of virtual characters that assist Human-Computer Interaction (HCI), e.g. as game opponents or personal assistants (PAs).
Endowing robots with a social interface is often costly and difficult. Virtual characters on the other hand are comparatively cheap and well equipped but suffer from other difficulties, most notably their inability to interact with the physical world. Combining physical robots and virtual characters into a mixed reality agent offers a cost-effective approach to enabling a rich social interface for human-robot interaction. A first incarnation of this principle is the use of a virtual character visualised on a screen placed on top of the robot. However, this approach suffers some crucial drawbacks as the screen can only be seen from one angle and the virtual character and its actions are still limited to the screen. 
