CITEULIKE: 6009995 | REFERENCE: BibTex, Endnote, RefMan | PDF PDF Version

Juarez, A., Bartneck, C., & Feijs, L. (2009). On the creation of standards for interaction between real robots and virtual worlds. Journal of Virtual Worlds Research, 2(3), 4-8.

On the Creation of Standards for Interaction Between Robots and Virtual Worlds

Alex Juarex, Christoph Bartneck, Loe Feijs

Department of Industrial Design
Eindhoven University of Technology
Den Dolech 2, 5600MB Eindhoven, NL,,

Abstract - Research on virtual worlds and environments has increased tremendously in the last decade, giving birth to a variety of applications spanning over several areas such as virtual reality, human-computer interaction, psychology and sociology, among others. In this paper we elaborate on one issue affecting the areas of virtual worlds and robotics: the lack of standard mechanisms for communication and interaction between virtual worlds and robots. We contribute our thoughts on the possibility of creating a standard platform that enable the seamless interaction between these heterogeneous, distributed devices and systems. Hopefully, these ideas will translate into applications that not only address the challenges in communication, control and interoperability of such systems (robots and virtual worlds), but also help to improve the standard of life of people through tangible products and services.

Keywords: Virtual Worlds, Robots, Robotics, Standards, Communication and Interaction

Robots and virtual worlds

Research on virtual worlds and environments has increased tremendously in the last decade, giving birth to a variety of applications spanning over several areas such as virtual reality, human-computer interaction, psychology and sociology, among others. We contribute our thoughts on the possibility of creating a standard platform that enable the seamless interaction between real robots and virtual worlds. Hopefully, these ideas will translate into applications that not only address the challenges in communication and interaction between such systems (robots and virtual worlds), but also help to improve the standard of life with tangible products and services.

Nowadays it is common to see humans of all ages subscribing to and using virtual worlds, an online representation of reality of the likes of those encountered in popular internet applications like Second Life ( or IMVU ( In these virtual worlds, humans can form communities and establish bonds with both avatars and other real people. Even more, the interaction is reaching levels where the real and virtual worlds merge: in “real-life” virtual items can be purchased on eBay and immediately be used in the virtual world. In a similar way, appliances and toys like the Nabaztag ( can detect events occurring in the virtual world and communicate them to their owners in the real world, showing a synergy that allows virtual and real agents to become essential parts of our lives.

One promising area of application for this kind of interaction is robotics. Traditionally, robots have been used to help humans in labor intensive and hazardous work, as research subjects or simply, as means of entertainment. The development in robotics has reached a high level of sophistication that can be easily appreciated in the many complex, precise and accurate manipulators, autonomous mobile platforms, surveillance and rescue vehicles, insect-like and humanoid robots available, either commercially or as research prototypes.

Yet, robots and robotics in general face a major challenge: to reach the masses. Many interesting and inspiring robotic projects do not reach media and public attention due to expensive components, poor performance on highly complex environments of operation, tight IP agreements, or simply because of bad marketing strategies.

The massive and growing popularity of virtual worlds is a characteristic that allows to showcase real robotic agents in challenging environments, showing their features in a collaborative setup, bringing them to mainstream attention and, even more importantly, reaching potential customers directly. Furthermore, virtual worlds allow us to test new robotic platforms in circumstances that most popular simulation tools lack: a highly interactive, non-deterministic, socially affected, close-to-reality environment, where the robot is able to show its true potential.

The social presence of a robot can also be increased with its inclusion in virtual environments. For example, a service robot that is able to connect to a virtual world can guide children or the elderly to interact and communicate with other people in the virtual environment, while monitoring them both in their real and virtual lives. This adds a social dimension to the task of the robot, making it useful to minimize loneliness, improve health and social care, and even providing some affection in the process (Nourbakhsh et al., 1999).
In addition, a robot that is designed, controlled and tested in a virtual environment offers the possibility of physically distant researchers to contribute to the creation of new prototypes in a more constructive, efficient and cost-effective way. The same environment can be easily used to commercialize the product by presenting it to potential customers in countries spread around the world, all at a fraction of the traditional investment in sales and marketing. The open nature of virtual environments, continuously connected to the internet, offers a huge potential to make the product known to larger audiences than those previously reached via more traditional advertisement mechanisms—and with a significant reduction in the associated costs.

It would be naive to say that the current level of development of virtual worlds offers a substitute for more traditional ways of developing, testing, commercializing and using a product. However, the rapid growth of virtual and mixed reality and the increasing interest of the research community and the general public can turn it into a viable economic alternative with which compete in a globalized world. In the next sections we present our thoughts on some of the current challenges that this research area offers, along with ideas on how to overcome them.

Fast Pace Development, Technical Isolation and Standardization

We believe that the exciting research and commercial opportunities offered by the integration of real robotic agents and popular virtual reality environments are hindered by the lack of standardization in the interaction between them. The fast pace of virtual worlds and robot technology development add a further aggravating component, which makes standard communication and interaction mechanisms more of a necessity than a simple feature of these systems.

Initial efforts in this area have tried to integrate tangible robotic spaces (a real robot and its surrounding environment) with a virtual world focusing on multiple-user robot control through avatars (Syamsuddin et al., 2008). Other approaches investigate the effects of social interaction and cooperation between humans and robots in scenarios that simulate reality, but are impractical to replicate in the real world (e.g. a simulation of potentially unsafe situations that can arise when humans and robots interact in a home environment) (Prattichizzo, 1999).

These approaches, however, are mostly technically isolated from one another, in the sense that the mechanisms that allow the interaction between the virtual environment (i.e. simulators, virtual worlds, etc.) and the real agents have been constructed in ad-hoc manner using heterogeneous technologies and, in some cases, neglecting the possibility of a conventional platform for their integration. In synthesis, most existing approaches do not concern themselves with one fundamental question: is it possible to build a common platform that allows the seamless integration (to a certain degree) of heterogeneous robotic hardware and virtual environments, such that the sensors and actuators can be monitored and controlled across software and hardware platforms?

We are convinced that it is not only possible, but necessary to produce such platforms that will allow the “next step" in the fusion of virtual and real worlds. Moreover, this platform can easily turn into a benchmark that allows researchers and industry to compare and judge the quality and performance of different hardware and software available.

Building a Standard for Communication and Interaction Between Virtual and Real Worlds

In order to build a standard for the interaction between real robots and virtual worlds, several challenges must be addressed:


Virtual worlds offer exciting opportunities for robotics, however they are currently hindered by the lack of a common platform where the heterogeneous robotic hardware and the different virtual environments available can integrate. We believe that the creation of a standardized mechanism for communication and interaction between real robots and virtual worlds is a crucial step in the development of the next generation technology and applications where robots can show their true potential. More concretely, this will allow us to build a general platform that can be used as a benchmark where researchers and industry can test and evaluate different software and hardware available.

We also anticipate that further development of this technology will provide interesting mechanisms to develop and test new products –in particular, robots - ensuring their usability, acceptability and reliability in different areas of application such as medical and health care, telerobotics, augmented and mixed reality.
Finally, the introduction of this technology into everyday life will allow the end user (the grandmother that lives alone at home, or the child that wants to meet with his friends living many kilometers apart) to experience a new form of social interaction: they will not isolate at home but instead they will be able to communicate to a real “friend,” a robot that can assist them.

In many cases this will result in a direct improvement in quality of life for many people. For example, for elderly people struggling with loneliness or illness, a robotic device can be used as a proxy to guide them in a journey through virtual worlds where they meet family and make new friends. At the same time, the robotic device can monitor their health and make sure that appropriate response is given in case of any emergency. This is a critical capability, as noted by T. G. Holzman (1999): “Quality medical care depends on prompt, accurate recording, communication, and retrieval of patient data […] In emergency medicine, such information can make the difference between life and death” (p. 13).


Holzman, T. G. (1999). Computer-human interface solutions for emergency medical care. Interactions ACM Journal, 6(3), 13-24.

Nourbakhsh, I.R., Bobenage, J., Grange, S., Lutz, R., Meyer, R., and Soto, A. (1999). An affective mobile robot educator with a full-time job. Artificial Intelligence, 114(1-2), 95–12.

Prattichizzo, D. (2009). Robotics in Second Life. IEEE Robotics and Automation Magazine, 16(1), 99-102.

Syamsuddin, M.R., Mayangsari, M.N., Juasiripukdee, P., and Kwon, Y.M. (2008). Trying to integrate ubiquitous robotic space and metaverse. In Proceedings of the Workshop on Virtual Worlds, Collaboration, and Workplace Productivity (CSCW08), San Diego, California.


This is a pre-print version | last updated October 26, 2009 | All Publications