Game testing

Last updated

Game testing, also called quality assurance (QA) testing within the video game industry, is a software testing process for quality control of video games. [1] [2] [3] The primary function of game testing is the discovery and documentation of software defects. Interactive entertainment software testing is a highly technical field requiring computing expertise, analytic competence, critical evaluation skills, and endurance. [4] [5] In recent years the field of game testing has come under fire for being extremely strenuous and unrewarding, both financially and emotionally. [6]

Contents

History

In the early days of computer and video games, the developer was in charge of all the testing. No more than one or two testers were required due to the limited scope of the games. In some cases, the programmers could handle all the testing.[ citation needed ]

As games become more complex, a larger pool of QA resources, called "Quality Assessment" or "Quality Assurance" is necessary. Most publishers employ a large QA staff for testing various games from different developers. Despite the large QA infrastructure most publishers have, many developers retain a small group of testers to provide on-the-spot QA.

Now most game developers rely on their highly technical and game savvy testers to find glitches and 'bugs' in either the programming code or graphic layers. Game testers usually have a background playing a variety of different games on a multitude of platforms. They must be able to notate and reference any problems they find in detailed reports, meet deadlines with assignments and have the skill level to complete the game titles on their most difficult settings. Most of the time the position of game tester is a highly stressful and competitive position with little pay yet is highly sought after for it serves as a doorway into the industry. Game testers are observant individuals and can spot minor defects in the game build.

A common misconception is that all game testers enjoy alpha or beta version of the game and report occasionally found bugs. [5] In contrast, game testing is highly focused on finding bugs using established and often tedious methodologies before alpha version.

Overview

Quality assurance is a critical component in game development, though the video game industry does not have a standard methodology. Instead developers and publishers have their own methods. Small developers do not generally have QA staff; however, large companies may employ QA teams full-time. High-profile commercial games are professionally and efficiently tested by publisher QA department. [7]

Testing starts as soon as first code is written and increases as the game progresses towards completion. [8] [9] The main QA team will monitor the game from its first submission to QA until as late as post-production. [9] Early in the game development process the testing team is small and focuses on daily feedback for new code. As the game approaches alpha stage, more team members are employed and test plans are written. Sometimes features that are not bugs are reported as bugs and sometimes the programming team fails to fix issues first time around. [10] A good bug-reporting system may help the programmers work efficiently. As the projects enters beta stage, the testing team will have clear assignments for each day. Tester feedback may determine final decisions of exclusion or inclusion of final features. Introducing testers with fresh perspectives may help identify new bugs. [9] [11] At this point the lead tester communicates with the producer and department heads daily. [12] If the developer has an external publisher, then coordination with publisher's QA team starts. For console games, a build for the console company QA team is sent. Beta testing may involve volunteers, for example, if the game is multiplayer. [11]

Testers receive scheduled uniquely identifiable game builds [11] from the developers.[ citation needed ] The game is play-tested and testers note any uncovered errors. These may range from bugs to art glitches to logic errors and level bugs. Testing requires creative gameplay to discover often subtle bugs. Some bugs are easy to document, but many require detailed description so a developer can replicate or find the bug. Testers implement concurrency control to avoid logging bugs multiple times.[ citation needed ] Many video game companies separate technical requirement testing from functionality testing altogether since a different testing skillset is required. [5]

If a video game development enters crunch time before a deadline, the game-test team is required to test late-added features and content without delay. During this period staff from other departments may contribute to the testing—especially in multiplayer games.[ citation needed ] One example of sustained crunch, especially among the QA team, was at Treyarch during the development of Call of Duty: Black Ops 4 . [13]

Most companies rank bugs according to an estimate of their severity: [14]

Game tester

A game tester is a member of a development team who performs game testing.

Roles

The organization of staff differs between organizations; a typical company may employ the following roles associated with testing disciplines:

Employment

Game QA is less technical than general software QA. Game testers most often require experience however occasionally only a high school diploma and with no technical expertise, suffice.[ citation needed ] Game testing is normally a full-time job for experienced testers; [18] however, many employees are hired as temporary staff, [2] [19] such as beta testers. In some cases, testers employed by a publisher may be sent to work at the developer's site. The most aggressive recruiting season is late summer/early autumn[ citation needed ], as this is the start of the crunch period for games to be finished and shipped in time for the holiday season.

Some games studios are starting to take a more technical approach to game QA that is more inline with traditional software testing. Technical Test positions are still fairly rare throughout the industry but these jobs are often full-time positions with long term career paths and require a 4-year computer science degree and significant experience with test automation.

Some testers use the job as a stepping stone in the game industry. [3] [20] QA résumés, which display non-technical skill sets, tend towards management, than to marketing or production.[ citation needed ] Applicants for programming, art, or design positions need to demonstrate technical skills in these areas. [21]

Compensation

Game testing personnel are usually paid hourly (around US$10–12 an hour). Testing management is usually more lucrative, and requires experience and often a college education. An annual survey found that testers earn an average of $39k annually. Testers with less than three years' experience earn an average of US$25k while testers with over three years' experience earn US$43k. Testing leads, with over six years' experience, earn on an average of US$71k a year. [22]

Process

A typical bug report progression of testing process is seen below:

Methodology

There is no standard method for game testing, and most methodologies are developed by individual video game developers and publishers. Methodologies are continuously refined and may differ for different types of games (for example, the methodology for testing an MMORPG will be different from testing a casual game). Many methods, such as unit testing, are borrowed directly from general software testing techniques. Outlined below are the most important methodologies, specific to video games.

The requirements are proprietary documents released to developers and publishers under confidentiality agreements. They are not available for the general public to review, although familiarity with these standards is considered a valuable skill to have as a tester.[ citation needed ]
Compliance may also refer to regulatory bodies such as the ESRB and PEGI, if the game targets a particular content rating. Testers must report objectionable content that may be inappropriate for the desired rating. Similar to licensing, games that do not receive the desired rating must be re-edited, retested, and resubmitted at additional cost.
Compatibility testing ensures that the game runs on different configurations of hardware and software. The hardware encompasses brands of different manufacturers and assorted input peripherals such as gamepads and joysticks.[ citation needed ]
The testers also evaluate performance and results are used for game's advertised minimum system requirements. Compatibility or performance issues may be either fixed by the developer or, in case of legacy hardware and software, support may be dropped.

Console hardware

For consoles, the majority of testing is not performed on a normal system or consumer unit. Special test equipment is provided to developers and publishers. The most significant tools are the test or debug kits, and the dev kits. The main difference from consumer units is the ability to load games from a burned disc, USB stick, or hard drive. The console can also be set to any publishing region. This allows game developers to produce copies for testing. This functionality is not present in consumer units to combat software piracy and grey-market imports.[ citation needed ]

See also

Notes

  1. Bates 2004, pp. 176-180
  2. 1 2 3 4 Moore, Novak 2010, p. 95
  3. 1 2 Oxland 2004, p. 301-302
  4. Bates 2004, pp. 178, 180
  5. 1 2 3 Oxland 2004, p. 301
  6. "The Tough Life of a Games Tester" from IGN
  7. 1 2 3 4 Bethke 2003, p. 52
  8. Bates 2004, p. 176
  9. 1 2 3 4 5 6 Bethke 2003, p. 53
  10. 1 2 3 4 Bates 2004, p. 177
  11. 1 2 3 4 5 6 7 Bates 2004, p. 178
  12. 1 2 3 4 Bates 2004, p. 179
  13. Brendan Sinclair (2019-06-26). "Stories of crunch, neglect for QA at Treyarch". gameindustry.biz. Retrieved 2022-06-09.
  14. 1 2 Bates 2004, pp. 178-179
  15. Moore, Novak 2010, p. 72
  16. Bob Johnstone. "Didi Games". Research on Video Games. Didi Games. Archived from the original on 2014-10-06. Retrieved 2009-04-01.
  17. Bates 2004, p. 180
  18. Moore, Novak 2010, p. 25
  19. Moore, Novak 2010, p. 2
  20. Bates 2004, p. 261
  21. Moore, Novak 2010, pp. 84, 237-238
  22. Fleming, Jeffrey (April 2008). "7th Annual Salary Survey". Game Developer . United Business Media. 15 (4): 8.
  23. Adams, Rollings 2003, p. 17
  24. Yannakakis, Geogios N (2012). "Game AI revisited". Proceedings of the 9th conference on Computing Frontiers (PDF). pp. 285–292. doi:10.1145/2212908.2212954. ISBN   9781450312158. S2CID   4335529. Archived (PDF) from the original on 8 August 2014.

Related Research Articles

<span class="mw-page-title-main">DirectX</span> Collection of multimedia related APIs on Microsoft platforms

Microsoft DirectX is a collection of application programming interfaces (APIs) for handling tasks related to multimedia, especially game programming and video, on Microsoft platforms. Originally, the names of these APIs all began with "Direct", such as Direct3D, DirectDraw, DirectMusic, DirectPlay, DirectSound, and so forth. The name DirectX was coined as a shorthand term for all of these APIs and soon became the name of the collection. When Microsoft later set out to develop a gaming console, the X was used as the basis of the name Xbox to indicate that the console was based on DirectX technology. The X initial has been carried forward in the naming of APIs designed for the Xbox such as XInput and the Cross-platform Audio Creation Tool (XACT), while the DirectX pattern has been continued for Windows APIs such as Direct2D and DirectWrite.

Raven Software Corporation is an American video game developer based in Wisconsin and founded in 1990. In 1997, Raven made an exclusive publishing deal with Activision and was subsequently acquired by them. After the acquisition, many of the studio's original developers, largely responsible for creating the Heretic and Hexen: Beyond Heretic games, left to form Human Head Studios.

Software testing is the act of examining the artifacts and the behavior of the software under test by verification and validation. Software testing can also provide an objective, independent view of the software to allow the business to appreciate and understand the risks of software implementation. Test techniques include, but are not limited to:

<span class="mw-page-title-main">Software release life cycle</span> Sum of the phases of development and maturity for computer software

The software release life cycle is the process of developing, testing, and distributing a software product. It typically consists of several stages, such as pre-alpha, alpha, beta, and release candidate, before the final version, or "gold", is released to the public.

A game programmer is a software engineer, programmer, or computer scientist who primarily develops codebases for video games or related software, such as game development tools. Game programming has many specialized disciplines, all of which fall under the umbrella term of "game programmer". A game programmer should not be confused with a game designer, who works on game design.

A console game is a type of video game consisting of images and often sounds generated by a video game console, which are displayed on a television or similar audio-video system, and that can be manipulated by a player. This manipulation usually takes place using a handheld device connected to the console, called a controller. The controller generally contains several buttons and directional controls such as analogue joysticks, each of which has been assigned a purpose for interacting with and controlling the images on the screen. The display, speakers, console, and controls of a console can also be incorporated into one small object known as a handheld game.

Video game design is the process of designing the rules and content of video games in the pre-production stage and designing the gameplay, environment, storyline and characters in the production stage. Some common video game design subdisciplines are world design, level design, system design, content design, and user interface design. Within the video game industry, video game design is usually just referred to as "game design", which is a more general term elsewhere.

A video game producer is the top person in charge of overseeing development of a video game.

Video game development is the process of creating a video game. It is a multidisciplinary practice, involving programming, design, art, audio, user interface, and writing. Each of those may be made up of more specialized skills; art includes 3D modeling of objects, character modeling, animation, visual effects, and so on. Development is supported by project management, production, and quality assurance. Teams can be many hundreds of people, a small group, or even a single person.

Game programming, a subset of game development, is the software development of video games. Game programming requires substantial skill in software engineering and computer programming in a given language, as well as specialization in one or more of the following areas: simulation, computer graphics, artificial intelligence, physics, audio programming, and input. For multiplayer games, knowledge of network programming is required. In some genres, e.g. fighting games, advanced network programming is often demanded, as the netcode and its properties are considered by players and critics to be some of the most important metrics of the game's quality. For massively multiplayer online games (MMOGs), even further knowledge of database programming and advanced networking programming are required. Though often engaged in by professional game programmers, there is a thriving scene of independent developers who lack a relationship with a publishing company.

A playtest is the process by which a game designer tests a new game for bugs and design flaws before releasing it to market. Playtests can be run "open", "closed", "beta", or otherwise, and are very common with board games, collectible card games, puzzle hunts, role-playing games, and video games, for which they have become an established part of the quality control process. An individual involved in testing a game is referred to as a playtester.

<span class="mw-page-title-main">PC game</span> Electronic game played on a personal computer

A personal computer game, also known as a computer game or abbreviated PC game, is an electronic game played on a personal computer (PC) and form of video game. They are defined by the open platform nature of PC systems.

In video games, a level is any space available to the player during the course of completion of an objective. Video game levels generally have progressively increasing difficulty to appeal to players with different skill levels. Each level may present new concepts and challenges to keep a player's interest high.

The art pipeline is the process of creating and implementing art for a particular project, most commonly associated with the creative process for developing video games. In an era of high-profile video games, wherein the creative energy of the teams and the budgets for projects surpass even some Hollywood blockbusters, graphics are ever-improving and an increasingly important selling point. Video Game developers employ extensive teams of artists to carry a project's artistic goals through from the conceptual stage to the final release. A fully realized game asset, whether it is a character, background, building, object, or animation, is created in a deliberate process with different artists working on and contributing separate aspects in a step by step process to the final product. To facilitate this process, a number of software programs are marketed to developers as flexible tools for productively streamlining the transformation of elements created within a program to a fully realized game asset. These highly customizable programs allow users to apply plug-ins or add-ons to tailor the interactions each program will have with one another as an element moves along the pipeline from concept to completion.

<span class="mw-page-title-main">Video game console emulator</span> Program that reproduces video game consoles behavior

A video game console emulator is a type of emulator that allows a computing device to emulate a video game console's hardware and play its games on the emulating platform. More often than not, emulators carry additional features that surpass limitations of the original hardware, such as broader controller compatibility, timescale control, easier access to memory modifications, and unlocking of gameplay features. Emulators are also a useful tool in the development process of homebrew demos and the creation of new games for older, discontinued, or rare consoles.

Game art design is a subset of game development involving the process of creating the artistic aspects of video games. Video game art design begins in the pre-production phase of creating a video game. Video game artists are visual artists involved from the conception of the game who make rough sketches of the characters, setting, objects, etc. These starting concept designs can also be created by the game designers before the game is moved into actualization. Sometimes, these concept designs are called "programmer art". After the rough sketches are completed and the game is ready to be moved forward, those artists or more artists are brought in to develop graphic designs based on the sketches.

<span class="mw-page-title-main">Crowdsourced Testing (company)</span> Crowdsourcing platform

Crowdsourced Testing is a crowdsourcing platform which provides functional, localization, usability and Beta testing through crowdsourcing.

A QA & UX Manager work with both Quality Assurance (QA) and User Experience (UX) in relation to video game and software development. QA & UX Manager can work independently or in co-operation with other QA & UX Managers, like in larger QA & UX teams with other QA & UX Managers and game testers. In the larger QA & UX teams, there is usually a lead QA & UX Group Manager that works as the daily leader of this team of QA & UX Managers and game testers. QA & UX Manager also usually work very close with the project managers and the QA Programmer as part of video game development.

This article discusses a set of tactics useful in software testing. It is intended as a comprehensive list of tactical approaches to Software Quality Assurance (more widely colloquially known as Quality Assurance and general application of the test method.

In the video game industry, games as a service (GaaS) represents providing video games or game content on a continuing revenue model, similar to software as a service. Games as a service are ways to monetize video games either after their initial sale, or to support a free-to-play model. Games released under the GaaS model typically receive a long or indefinite stream of monetized new content over time to encourage players to continue paying to support the game. This often leads to games that work under a GaaS model to be called "living games", "live games", or "live service games" since they continually change with these updates.

References

Research