Pointman (user interface)

Last updated
Pointman: A User Interface for Dismounted Infantry Simulation
Developer(s) US Naval Research Laboratory
Platform Microsoft Windows
Type Military simulation and training. Computer simulation.

Pointman is a seated user interface for controlling one's avatar in a 3D virtual environment. It combines head tracking, a gamepad, and sliding foot pedals to provide positional control over many aspects of the avatar's posture. [1] [2] Pointman was developed by the US Naval Research Laboratory (NRL) to support the use of dismounted infantry simulation for USMC training and mission rehearsal. [1] [2] [3] NRL's goal in developing Pointman was to extend the range and precision of actions supported by virtual simulators, to better represent what infantrymen can do. [4]

Contents

How Pointman Works

Pointman seeks to enhance the level of control provided by conventional desktop and console game controllers by engaging the user's whole body to control corresponding segments of the avatar's body. [1] [2] [3] [5] The user employs his head and upper body to control looking and aiming, as well as leaning to duck and peek around cover. He uses his hands to operate virtual weapons and direct tactical movement, and he uses his feet for stepping and controlling his avatar's postural height. [1] [2]

Pointman uses a set of three consumer grade input devices: a Natural Point TrackIR 5 head tracker, a Sony DualShock 3 gamepad, and a pair of flight simulator foot pedals from CH Products. [1] The additional input from the head and feet offloads the hands from having to control the entire avatar and allows for a more natural assignment of control. Together, the three input devices offer twelve independent channels of control over the avatar's posture. [4]

The head tracker registers the translation and rotation of the user's head. Pointman uses these inputs to map the movements of the user's head and torso one-for-one to those of his avatar. [1] [2] The virtual view changes as the user turns his head to look around or leans his torso in any direction. When the weapon is raised into an aim position its sights remain centered in the field of view, so that turning the head also adjusts the aim. [1] The user can aim as precisely as he can hold his head on target. [2] [6] Hunching the head down by flexing the spine is also registered by the head's translation, and the avatar adopts a matching posture. Leaning forward and hunching are used to duck behind cover. Rising up and leaning to the side are used to look out and shoot from behind cover. [1] [2]

The gamepad includes dual thumb sticks and a pair of tilt sensors. Pointman uses the thumb sticks to turn the avatar's body and set the stepping direction. [1] [2] [3] The tilt of the gamepad is mapped to control how the virtual rifle is held. The user tilts the gamepad down to lower the rifle, and tilts the gamepad up to continuously raise the rifle up through a low ready into an aim and then to a high ready. [1] [2] This allows users to practice muzzle discipline, by lowering the rifle to avoid muzzle sweeping friendlies, minimizing collisions when moving through tight spaces, or leading with the rifle when moving around cover. [2] [6] Once the rifle is raised into an aim position, the user's head motion aligns the sight picture. [1] [2] The user rolls the gamepad (tilting it side to side) to cant the weapon. [1] Gamepad buttons are mapped to control various weapon operations (including firing and reloading) and aiming functions (such as the optic zoom level). [7] The foot pedals slide back and forth and also move up and down like accelerator pedals. Pointman uses these inputs to control the avatar's lower body. [1] [2] [3] The translational sliding (apart then together) is mapped to control the separation of the avatar's legs, simulating stepping when the avatar is upright and crawling when the avatar is prone. [2] This allows users to take precise, measured steps when moving around obstacles or cover, and to continuously vary their speed over a realistic range of walking, running and crawling gaits. [1] [2] [6] The up-down movement of the pedals is mapped to control the avatar's postural height via the flexing of the avatar's legs. This allows the avatar to continuously transition from standing tall to a low crouch (or kneel when the legs are apart), and when prone from hands-and-knees to belly-on-the-ground. [1] [2] The ability to precisely control their avatar's postural height allows users to make better use of cover and concealment, and to look and shoot out from behind cover while minimizing their exposure. [2] [3] [6]

Integration with the Virtual Battlespace Combined Arms Simulator

The Virtual Battlespace combined arms simulators (VBS2, VBS3, and their successor VBS4), from Bohemia Interactive Simulations (BIS), are used for training by the USMC, the US Army, and a number of the NATO armed forces. BIS worked closely with NRL to tightly integrate the Pointman interface with VBS, and to allow Pointman to control the posture of the user's avatar on a continuous basis. [1] [2] The detailed articulation of the user's avatar is made visible to other squad members running in a networked simulation. Pointman-enhanced VBS (VBS-Pointman) supports the operation of a wide range of small arms and additional forms of mobility, including climbing, swimming, and mounted roles (driver, passenger and gunner) using the full complement of manned vehicles. [8]

Development and Assessment

Pointman was designed by Dr. Jim Templeman of NRL and implemented by Patricia Denbrook of DCS, Inc. NRL's Base Funding Program supported the initial development of Pointman, and the USMC Program Manager Training Systems (PM_TRASYS) together with the Office of Naval Research (ONR) Rapid Technology Transition program office supported its integration with VBS2. ONR's Human Performance, Training, and Education Thrust Area added its support in refining and demonstrating Pointman. [1]

A formal Military Utility Assessment (MUA) of Pointman integrated with VBS2 was performed by the MarForPac Experimentation Center at MCB Hawaii in September 2011. [1] [2] [5] [6] The squad of Marines that participated in the study (Golf Company, 2nd Battalion, 34d Marine Regiment) gave Pointman high marks for realism and usability. In response to a series of survey questions, the Marines felt Pointman allowed them to realistically: control viewing, perform tactical movements, control the virtual rifle, utilize cover, and control the avatar’s posture. They found it comfortable, easy to use, and that it enhanced the simulation. The primary recommendation of the MUA report was: “Transition the Pointman DISI (dismounted infantry simulation interface) enhancements into VBS2 to increase realism and efficacy as a virtual training aid.” [6]

Future Enhancements

NRL is continuing[ when? ] to develop the Pointman interface as part of its ongoing research in expressive interaction for desktop simulation. This involves extending Pointman to include non-verbal communications (such as eye movements, facial expression, and arm gestures) needed to support team and cross-cultural interaction, without limiting tactical mobility. [4] A driving application is the training of cultural interaction skills alongside warfighting skills, using training scenarios which pose a mix of tactical, cultural and ethical challenges. [9]

Related Research Articles

<span class="mw-page-title-main">Graphical user interface</span> User interface allowing interaction through graphical icons and visual indicators

A graphical user interface, or GUI, is a form of user interface that allows users to interact with electronic devices through graphical icons and audio indicators such as primary notation. In many applications, GUIs are used instead of text-based UIs, which are based on typed command labels or text navigation. GUIs were introduced in reaction to the perceived steep learning curve of command-line interfaces (CLIs), which require commands to be typed on a computer keyboard.

<span class="mw-page-title-main">Flight simulator</span> Technology used for training aircrew

A flight simulator is a device that artificially re-creates aircraft flight and the environment in which it flies, for pilot training, design, or other purposes. It includes replicating the equations that govern how aircraft fly, how they react to applications of flight controls, the effects of other aircraft systems, and how the aircraft reacts to external factors such as air density, turbulence, wind shear, cloud, precipitation, etc. Flight simulation is used for a variety of reasons, including flight training, the design and development of the aircraft itself, and research into aircraft characteristics and control handling qualities.

<span class="mw-page-title-main">Game controller</span> Device used with games or entertainment systems

A game controller, gaming controller, or simply controller, is an input device or input/output device used with video games or entertainment systems to provide input to a video game. Input devices that have been classified as game controllers include keyboards, mouses, gamepads, and joysticks, as well as special purpose devices, such as steering wheels for driving games and light guns for shooting games. Controllers designs have evolved to include directional pads, multiple buttons, analog sticks, joysticks, motion detection, touch screens and a plethora of other features.

<span class="mw-page-title-main">Tactical shooter</span> Video game terminology

Tactical shooter is a sub-genre of first- and third-person shooters, associated with using strategy, planning, and tactics in gameplay, as well as the realistic simulations of ballistics, accurate firearm mechanics, physics and stamina, and low time to kill. Dating back to the late 1980s, the genre first rose to prominence in the late 1990s with the releases of several well-received tactical shooters. The popularity of the genre saw a decline in the late 2000s as more the arcade-like fast-paced action of other types of shooters rose to prominence, but it has seen a revitalization since the mid-2010s with the successful releases of several modern tactical shooters.

<span class="mw-page-title-main">Telerobotics</span>

Telerobotics is the area of robotics concerned with the control of semi-autonomous robots from a distance, chiefly using television, wireless networks or tethered connections. It is a combination of two major subfields, which are teleoperation and telepresence.

X-Plane is a flight simulation software initially launched by Laminar Research in 1995. Commercial desktop versions are sold for macOS, Windows, and Linux. In addition, Laminar Research also distributes FAA-certified versions for professional use. A mobile version has been available for Android, iOS, and webOS since 2009 as well.

An application program is a computer program designed to carry out a specific task other than one relating to the operation of the computer itself, typically to be used by end-users. Word processors, media players, and accounting software are examples. The collective noun "application software" refers to all applications collectively. The other principal classifications of software are system software, relating to the operation of the computer, and utility software ("utilities").

<span class="mw-page-title-main">Simulation cockpit</span> Cockpit used for training pilots with a flight simulator

A simulation cockpit, simpit or sim rig is an environment designed to replicate a vehicle cockpit. Although many pits commonly designed around an aircraft cockpit, the term is equally valid for train, spacecraft or car projects.

<span class="mw-page-title-main">Motion simulator</span> Type of mechanism

A motion simulator or motion platform is a mechanism that creates the feelings of being in a real motion environment. In a simulator, the movement is synchronised with a visual display of the outside world (OTW) scene. Motion platforms can provide movement in all of the six degrees of freedom (DOF) that can be experienced by an object that is free to move, such as an aircraft or spacecraft:. These are the three rotational degrees of freedom and three translational or linear degrees of freedom.

<span class="mw-page-title-main">Maritime simulator</span> Environment simulating a ship

A maritime simulator or ship simulator is a system that simulates ships and maritime environments for training, research and other purposes. Today, simulator training given by maritime schools and academies is part of the basic training of maritime professionals.

<span class="mw-page-title-main">Immersion (virtual reality)</span> Perception of being physically present in a non-physical world

Immersion into virtual reality (VR) is a perception of being physically present in a non-physical world. The perception is created by surrounding the user of the VR system in images, sound or other stimuli that provide an engrossing total environment.

The Deployable Virtual Training Environment (DVTE) is a laptop based platform for a wide variety of training simulations in the USMC. TRASYS web. DVTE consists of two main components, the Combined Arms Network (CAN) and the Infantry Tool Kit (ITK). A typical DVTE suite consists of 32 laptops and a modified NVisor SX Head-mounted display (HMD).

An instructional simulation, also called an educational simulation, is a simulation of some type of reality but which also includes instructional elements that help a learner explore, navigate or obtain more information about that system or environment that cannot generally be acquired from mere experimentation. Instructional simulations are typically goal oriented and focus learners on specific facts, concepts, or applications of the system or environment. Today, most universities make lifelong learning possible by offering a virtual learning environment (VLE). Not only can users access learning at different times in their lives, but they can also immerse themselves in learning without physically moving to a learning facility, or interact face to face with an instructor in real time. Such VLEs vary widely in interactivity and scope. For example, there are virtual classes, virtual labs, virtual programs, virtual library, virtual training, etc. Researchers have classified VLE in 4 types:

<span class="mw-page-title-main">Input device</span> Device that provides data and signals to a computer

In computing, an input device is a piece of equipment used to provide data and control signals to an information processing system, such as a computer or information appliance. Examples of input devices include keyboards, mouse, scanners, cameras, joysticks, and microphones.

<span class="mw-page-title-main">Flight simulation video game</span> Video game genre

A flight simulation video game refers to the simulation of various aspects of flight or the flight environment for purposes other than flight training or aircraft development. A significant community of simulation enthusiasts is supported by several commercial software packages, as well as commercial and homebuilt hardware. Open-source software that is used by the aerospace industry like FlightGear, whose flight dynamics engine (JSBSim) is used in a 2015 NASA benchmark to judge new simulation code to space industry standards, is also available for private use. A popular type of flight simulators video games are combat flight simulators, which simulate combat air operations from the pilot and crew's point of view. Combat flight simulation titles are more numerous than civilian flight simulators due to variety of subject matter available and market demand.

<span class="mw-page-title-main">Simulator ride</span>

Simulator rides are a type of amusement park or fairground ride, where the audience is shown a movie while their seats move to correspond to the action on screen.

The DiSTI Corporation is a company that provides software tools for the development of GUI software and 3D virtual training for simulators and embedded systems.

The virtual world framework (VWF) is a means to connect robust 3D, immersive, entities with other entities, virtual worlds, content and users via web browsers. It provides the ability for client-server programs to be delivered in a lightweight manner via web browsers, and provides synchronization for multiple users to interact with common objects and environments. For example, using VWF, a developer can take video lesson plans, component objects and avatars and successfully insert them into an existing virtual or created landscape, interacting with the native objects and users via a VWF interface.

Military Open Simulator Enterprise Strategy (MOSES) is a U.S. Army project evaluating the ability of OpenSimulator to provide independent and secured access to a virtual world.

<span class="mw-page-title-main">Full motion racing simulator</span>

A full motion racing simulator, sometimes called a full motion sim rig, is a motion simulator that is purposed for racing, and must provide motion simulation in all six degrees of freedom, as defined by the aviation simulator industry many decades ago. The six degrees of freedom coincide with Earth physics, and are commonly referred to as:

References

  1. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 Templeman, J.; Denbrook, P. (2012). "Enhancing Realism in Desktop Interfaces for Dismounted Infantry Simulation". Interservice/Industry Training, Simulation, and Education Conference (I/ITSEC).
  2. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 Templeman, J.; Denbrook, P. (2012). "Pointman: a User Interface for Dismounted Infantry Simulation". Proceedings of the International Summit on Human Simulation: 95–105.
  3. 1 2 3 4 5 Templeman, J.N.; Sibert, L.E.; Page, R.C.; Denbrook, P.S. (2009). "Pointman Turning Virtual Reality into a More Realistic and Effective Infantry Training Ground". NRL Review: 60–65.
  4. 1 2 3 Templeman, J. "Enhancing Realism in Desktop Interfaces for Dismounted Infantry Simulation".
  5. 1 2 Biron, Lauren. "Pointman Adds Head, Leg Motion to PC Sim". Defense News. Archived from the original on May 8, 2014. Retrieved 2 January 2013.
  6. 1 2 3 4 5 6 Office of Naval Research, Code 30: Expeditionary Maneuver Warfare & Combating Terrorism Dept. (2012). "Pointman Dismounted Infantry Simulation Interface Military Utility Assessment Report".{{cite journal}}: Cite journal requires |journal= (help)CS1 maint: numeric names: authors list (link)
  7. ↑ Denbrook, P. (March 2014). "Quick Reference Cards". VBS-Pointman Training Package (Pointman Version 3.1.034).
  8. ↑ Denbrook, P. (March 2014). "Training Guide". VBS-Pointman Training Package (Pointman Version 3.1.034).
  9. ↑ "Marine Corps Vision & Strategy 2025". Office of Naval Research.
This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.