History
OpenIllusionist is closely connected with the Media Engineering Group (MEG) of the Department of Electronics at the University of York, UK - specifically the Visual Systems subgroup. This group was formed when John Robinson took up a professorship in the Department in 2000/2001, bringing with him a background in image coding and an interest in augmented reality.
This manifested itself in the work of three undergraduates - Dan Parnham, who experimented with the interpretation of the pose of a mannequin by the use of a single webcam focussed exclusively on the input side of the augmentation problem during his Master's degree, Sean O'Mahoney, who developed the first incarnation what would come to be termed PenPets as his Masters project, and Enrico Costanza, who developed a variety of tangible augmented interfaces using fiducials stuck to blocks of wood, with augmentation provided variously by audio feedback ("Audio d-Touch"), and a projector ("Magic Desk"). Much of Audio d-Touch was created by Enrico in his spare time as a personal project, (with collaboration from the aforementioned Robinson, and Simon Shelley, another York alumni) while the Magic Desk became his Masters project.
All of these projects fed into a collective culture in the group - with Justen Hyde, then a research student studying the reconstruction of human facial images, getting sucked into the work, making minor contributions to all of the projects, though officially working on none of them. The projects most often wheeled out to demonstrations were quickly established as PenPets (O'Mahoney) and d-Touch (Costanza) both of which had a strong commonality - they appeared to work by magic. The computer could be hidden from view, and the user could then simply interact with the augmentation directly. In the case of d-Touch, by moving marked blocks in front of a webcam to sample, edit and produce music with very low-cost paraphernalia - just a cheap microphone, printed fiducials and a standard PC. PenPets required more of a hardware overhead - a data projector pointed at a table. Onto this agents resembling mice were projected. These would run around the table, bouncing off hands, pen marks and objects.
After O'Mahoney and Costanza left the group, in 2002 and 2003 respectively, d-Touch continued to be developed by Costanza, but PenPets remained only a proof-of-concept, not technically beyond the prototype stage, and was mothballed. Parnham and Hyde continued to work on other aspects of image processing. However, the desire to work on a VAE was far from lost... Funding, as ever, was the only stumbling block, along with the unsuitability of the PenPets code for further development and expansion.
In 2004, a new Centre for Usable Home Technology was launched at York. As part of the launch event, a virtual augmented environment was promised - but upon inspection the PenPets demo, which had been considered, was found to be unreliable outside a laboratory environment, and almost impossible to maintain with no developer available with any experience with the code. In order to provide a demo, Hyde and Parnham, in their own time, went away and using all that had been learned in the years of development of the various group project, designed and built a new interactive aLife demo from scratch in the space of a week. As PenPets had never made it beyond experimental prototype stage, they decided to bite the bullet and rather than building just a demo instead built a basic but extensible generic augmented reality framework upon which an aLife demo could be run. This framework became the core of the OpenIllusionist Project.
Over the latter half of 2004, interest in the framework put together for this demo escalated, and the advantages of maintaining a framework upon which VAE development could be carried out became readily apparent. Instead of weeks to build a stable VAE demo application, results could be achieved in hours. The extensible structure of the proto-Illusionist meant that applications completely unlike the original aLife demo were being supported with relative ease. In the Autumn of 2004, it was decided that the framework was more useful and important than any given implementation of a demo on that framework - and also that an opportunity presented itself to begin the push to get augmented desktops out of research labs, and into circulation with the general public. The demo framework became an entity in its own right - the Illusionist - and was published as open source software.
Since then, development has continued, with the project still administered and run by the two founders, Dan Parnham and Justen Hyde. In 2006 "Robot Ships", an exhibit built using OpenIllusionist, was installed in the new Connect gallery at the National Museum of Scotland.
Augmented reality (AR) is an interactive experience that combines the real world and computer-generated content. The content can span multiple sensory modalities, including visual, auditory, haptic, somatosensory and olfactory. AR can be defined as a system that incorporates three basic features: a combination of real and virtual worlds, real-time interaction, and accurate 3D registration of virtual and real objects. The overlaid sensory information can be constructive, or destructive. This experience is seamlessly interwoven with the physical world such that it is perceived as an immersive aspect of the real environment. In this way, augmented reality alters one's ongoing perception of a real-world environment, whereas virtual reality completely replaces the user's real-world environment with a simulated one.
A head-up display, or heads-up display, also known as a HUD, is any transparent display that presents data without requiring users to look away from their usual viewpoints. The origin of the name stems from a pilot being able to view information with the head positioned "up" and looking forward, instead of angled down looking at lower instruments. A HUD also has the advantage that the pilot's eyes do not need to refocus to view the outside after looking at the optically nearer instruments.
A cave automatic virtual environment is an immersive virtual reality environment where projectors are directed to between three and six of the walls of a room-sized cube. The name is also a reference to the allegory of the Cave in Plato's Republic in which a philosopher contemplates perception, reality, and illusion.
Mixed reality (MR) is a term used to describe the merging of a real-world environment and a computer-generated one. Physical and virtual objects may co-exist in mixed reality environments and interact in real time. Mixed reality is largely synonymous with augmented reality.
A handheld projector is an image projector in a handheld device. It was developed as a computer display device for compact portable devices such as mobile phones, personal digital assistants, and digital cameras, which have sufficient storage capacity to handle presentation materials but are too small to accommodate a display screen that an audience can see easily. Handheld projectors involve miniaturized hardware, and software that can project digital images onto a nearby viewing surface.
NLS, or the "oN-Line System", was a revolutionary computer collaboration system developed in the 1960s. Designed by Douglas Engelbart and implemented by researchers at the Augmentation Research Center (ARC) at the Stanford Research Institute (SRI), the NLS system was the first to employ the practical use of hypertext links, the mouse, raster-scan video monitors, information organized by relevance, screen windowing, presentation programs, and other modern computing concepts. It was funded by ARPA, NASA, and the US Air Force.
A tangible user interface (TUI) is a user interface in which a person interacts with digital information through the physical environment. The initial name was Graspable User Interface, which is no longer used. The purpose of TUI development is to empower collaboration, learning, and design by giving physical forms to digital information, thus taking advantage of the human ability to grasp and manipulate physical objects and materials.
A fiducial marker or fiducial is an object placed in the field of view of an imaging system that appears in the image produced, for use as a point of reference or a measure. It may be either something placed into or on the imaging subject, or a mark or set of marks in the reticle of an optical instrument.
The Reactable is an electronic musical instrument with a tabletop tangible user interface that was developed within the Music Technology Group at the Universitat Pompeu Fabra in Barcelona, Spain by Sergi Jordà, Marcos Alonso, Martin Kaltenbrunner and Günter Geiger.
Surface computing is the use of a specialized computer GUI in which traditional GUI elements are replaced by intuitive, everyday objects. Instead of a keyboard and mouse, the user interacts with a surface. Typically the surface is a touch-sensitive screen, though other surface types like non-flat three-dimensional objects have been implemented as well. It has been said that this more closely replicates the familiar hands-on experience of everyday object manipulation.
A projection augmented model is an element sometimes employed in virtual reality systems. It consists of a physical three-dimensional model onto which a computer image is projected to create a realistic looking object. Importantly, the physical model is the same geometric shape as the object that the PA model depicts.
Augmented learning is an on-demand learning technique where the environment adapts to the learner. By providing remediation on-demand, learners can gain greater understanding of a topic while stimulating discovery and learning.
Wikitude is a mobile augmented reality (AR) technology provider based in Salzburg, Austria. Founded in 2008, Wikitude initially focused on providing location-based augmented reality experiences through the Wikitude World Browser App. In 2012, the company restructured it's proposition by launching the Wikitude SDK, a development framework utilizing image recognition and tracking, and geolocation technologies.
A virtual touch screen (VTS) is a user interface system that augments virtual objects into reality either through a projector or optical display using sensors to track a person's interaction with the object. For instance, using a display and a rear projector system a person could create images that look three-dimensional and appear to float in midair. Some systems utilize an optical head-mounted display to augment the virtual objects onto the transparent display utilizing sensors to determine visual and physical interactions with the virtual objects projected.
Projection mapping, similar to video mapping and spatial augmented reality, is a projection technique used to turn objects, often irregularly shaped, into display surfaces for video projection. The objects may be complex industrial landscapes, such as buildings, small indoor objects, or theatrical stages. Using specialized software, a two- or three-dimensional object is spatially mapped on the virtual program which mimics the real environment it is to be projected on. The software can then interact with a projector to fit any desired image onto the surface of that object. The technique is used by artists and advertisers who can add extra dimensions, optical illusions, and notions of movement onto previously static objects. The video is commonly combined with or triggered by audio to create an audiovisual narrative. In recent years the technique has also been widely used in the context of cultural heritage, as it has proved to be an excellent edutainment tool.
Vuforia is an augmented reality software development kit (SDK) for mobile devices that enables the creation of augmented reality applications. It uses computer vision technology to recognize and track planar images and 3D objects in real time. This image registration capability enables developers to position and orient virtual objects, such as 3D models and other media, in relation to real world objects when they are viewed through the camera of a mobile device. The virtual object then tracks the position and orientation of the image in real-time so that the viewer's perspective on the object corresponds with the perspective on the target. It thus appears that the virtual object is a part of the real-world scene.
An ARTag is a fiducial marker system to support 3D registration (alignment) and pose tracking in augmented reality. They can be used to facilitate the appearance of virtual objects, games, and animations within the real world. Like the earlier ARToolKit system, they allow for video tracking capabilities that calculate a camera's position and orientation relative to physical markers in real time. Once the camera's position is known, a virtual camera can be positioned at the same point, revealing the virtual object at the location of the ARTag. It thus addresses two of the key problems in Augmented Reality: viewpoint tracking and virtual object interaction.
IllumiRoom is a Microsoft Research project that augments a television screen with images projected onto the wall and surrounding objects. The current proof-of-concept uses a Kinect sensor and video projector. The Kinect sensor captures the geometry and colors of the area of the room that surrounds the television, and the projector displays video around the television that corresponds to a video source on the television, such as a video game or movie.
Toby L. J. Howard is an Honorary Reader in the Department of Computer Science at the University of Manchester in the UK, He was Director of undergraduate studies 2011–2019. He retired from the University in July 2020.
The Augmented Reality Sandtable (ARES) is an interactive, digital sand table that uses augmented reality (AR) technology to create a 3D battlespace map. It was developed by the Human Research and Engineering Directorate (HRED) at the Army Research Laboratory (ARL) to combine the positive aspects of traditional military sand tables with the latest digital technologies to better support soldier training and offer new possibilities of learning. It uses a projector to display a topographical map on top of the sand in a regular sandbox as well as a motion sensor that keeps track of changes in the layout of the sand to appropriately adjust the computer-generated terrain display.
