AlphaStar (software)

Last updated

AlphaStar is an artificial intelligence (AI) software developed by DeepMind for playing the video game StarCraft II . It was unveiled to the public by name in January 2019. AlphaStar attained "Grandmaster" status in August 2019, a significant milestone not just for AI in video games, but arguably for the field as a whole.

Contents

Background

Games created for humans are considered to have external validity as benchmarks of progress in artificial intelligence. IBM's chess engine Deep Blue (1997) and DeepMind's AlphaGo (2016) were considered major milestones; some argue that StarCraft would also be a major milestone, due to the game's "real-time play, partial observability, no single dominant strategy, complex rules that make it hard to build a fast forward model, and a particularly large and varied action space." [1] Though difficult, StarCraft may still be tractable with current technology because "its rules are known and the world is discrete with only a few types of objects". [2]

StarCraft II is a popular fast-paced online real-time strategy game developed by Blizzard Entertainment. [3] [4]

History

DeepMind Technologies was founded in the UK in 2010. As early as 2011, founder Demis Hassabis called StarCraft "the next step up" after games like Go. [5] DeepMind became a subsidiary of Google in 2014, after demonstrating self-learning bots with superhuman ability at a variety of Atari 2600 games. In February 2015, computer scientist Zachary Mason predicted Deepmind's research "leads to StarCraft in five or ten years". [6] In March 2016, following AlphaGo's victory over Lee Sedol, a world champion Go player, Hassabis publicly mulled building an AI for StarCraft, citing it as a strategic game with incomplete information where, unlike Go, much of the "board" is invisible. [7] [8] A formal collaboration was announced at BlizzCon in November 2016, alongside a plan to release an open development environment for bots in Q1 of 2017. [9]

By 2017, DeepMind was experimenting with feeding StarCraft data into its software. In August 2017, DeepMind and Blizzard released development tools to assist in bot development, as well as data from 65,000 historical games. At the time, computer scientist and StarCraft tournament manager David Churchill estimated it would take five years for a bot to beat a human, but made the caveat that AlphaGo had beaten expectations. In Wired , tech journalist Tom Simonite stated "No one expects the robot to win anytime soon. But when it does, it will be a far greater achievement than DeepMind's conquest of Go." [10]

In December 2018, DeepMind's bot defeated professional player Grzegorz "MaNa" Komincz, 5-0. DeepMind announced the bot, named "AlphaStar", in January 2019. A journalist at Ars Technica and others argued that AlphaStar still had unfair advantages: "AlphaStar has the ability to make its clicks with surgical precision using an API, whereas human players are constrained by the mechanical limits of computer mice". AlphaStar also had a global view rather than being limited by the in-game camera. Furthermore, while there was a cap on the number of actions over a five-second window, AlphaStar was free to allocate its action quota unevenly across the window in order to launch superhuman bursts of activity at critical moments. DeepMind quickly retrained AlphaStar under more realistic constraints, and then lost a rematch with Komincz. [11] Starting in July 2019, the new, constrained version of AlphaStar anonymously competed against players who "opted in" on the public 1v1 European multiplayer ladder. [12] By the end of August 2019, AlphaStar had attained Grandmaster level, ranking among the top 0.2% of human players. [13]

Algorithms

Unlike AlphaZero, AlphaStar initially learns to imitate the moves of the best players in its database of human vs. human games; this step is necessary to solve what DeepMind's Dave Silver calls "the exploration problem": discovering new strategies would otherwise be like finding a "needle in a haystack". Agents then play each other and deploy deep reinforcement learning. These main agents also learn by playing against suboptimal "exploiter agents" whose purpose is to expose weaknesses in the main agents. [14] [15]

Reactions

After his 5-0 defeat in December 2018, Komincz stated "I wasn't expecting the AI to be that good". [16]

Stuart Russell assessed that AlphaStar's 2018 victory required "a fair amount of problem-specific effort" and that general-purpose methods were "not quite ready for StarCraft". [2]

An article in Wired UK judged AlphaStar's new constraints, adopted for the July 2019 matches, to be "fair" this time around. [17] StarCraft professional Raza "RazerBlader" Sekha stated AlphaStar was "impressive" but had its quirks, succumbing in one game to an unorthodox army composition made up of only air units. The UK's top player, Joshua "RiSky" Hayward, expressed some disappointment, saying AlphaStar "often didn't make the most efficient, strategic decisions". [14] Professional Diego "Kelazhur" Schwimer called AlphaStar's play "unimaginably unusual; it really makes you question how much of StarCraft's diverse possibilities pro players have really explored". [13] AlphaStar's opponents often did not realize they were playing a bot. [18]

Ian Sample, of The Guardian , called AlphaStar a "landmark achievement" for the field of AI. [19] Churchill stated that he had previously seen bots that master one or two elements of StarCraft, but that AlphaStar was the first that can handle the game in its entirety. [3] Gary Marcus expressed his continuing skepticism about deep learning, stating: "So far the field has struggled to take techniques like this out of the laboratory and game environments and into the real world, and I don't immediately see this result as progress in that direction". [14] AI researcher Jon Dodge was surprised by AlphaStar, stating that he did not expect such a "superhuman" performance for "another couple of years"; in contrast, Churchill states "StarCraft is nowhere near being 'solved', and AlphaStar is not yet even close to playing at a world champion level". [4]

Legacy

DeepMind argues that insights from AlphaStar might benefit robots, self-driving cars, and virtual assistants, which need to operate with "imperfectly observed information". Silver has indicated his lab "may rest at this point", rather than try to substantially improve AlphaStar. [14] Silver himself argues that "AlphaStar has become the first AI system to reach the top tier of human performance in any professionally played e-sport on the full unrestricted game under professionally approved conditions... Ever since computers cracked Go, chess, and poker, the game of StarCraft has emerged, essentially by consensus from the community, as the next grand challenge for AI." [19]

Computer scientist Noel Sharkey argues, disapprovingly, that "military analysts will certainly be eyeing the successful AlphaStar real-time strategies as a clear example of the advantages of AI for battlefield planning". [19] In contrast, Silver argues: "To say that this has any kind of military use is saying no more than to say an AI for chess could be used to lead to military applications". [14]

See also

Related Research Articles

<span class="mw-page-title-main">Deep Blue (chess computer)</span> Chess-playing computer made by IBM

Deep Blue was a chess-playing expert system run on a unique purpose-built IBM supercomputer. It was the first computer to win a game, and the first to win a match, against a reigning world champion under regular time controls. Development began in 1985 at Carnegie Mellon University under the name ChipTest. It then moved to IBM, where it was first renamed Deep Thought, then again in 1989 to Deep Blue. It first played world champion Garry Kasparov in a six-game match in 1996, where it lost four games to two. It was upgraded in 1997 and in a six-game re-match, it defeated Kasparov by winning two games and drawing three. Deep Blue's victory is considered a milestone in the history of artificial intelligence and has been the subject of several books and films.

<span class="mw-page-title-main">Computer Go</span> Field of artificial intelligence around Go computer programs

Computer Go is the field of artificial intelligence (AI) dedicated to creating a computer program that plays the traditional board game Go. The field is sharply divided into two eras. Before 2015, the programs of the era were weak. The best efforts of the 1980s and 1990s produced only AIs that could be defeated by beginners, and AIs of the early 2000s were intermediate level at best. Professionals could defeat these programs even given handicaps of 10+ stones in favor of the AI. Many of the algorithms such as alpha-beta minimax that performed well as AIs for checkers and chess fell apart on Go's 19x19 board, as there were too many branching possibilities to consider. Creation of a human professional quality program with the techniques and hardware of the time was out of reach. Some AI researchers speculated that the problem was unsolvable without creation of human-like AI.

<span class="mw-page-title-main">Demis Hassabis</span> British artificial intelligence researcher (born 1976)

Sir Demis Hassabis is a British artificial intelligence (AI) researcher, and entrepreneur. He is the chief executive officer and co-founder of Google DeepMind, and Isomorphic Labs, and a UK Government AI Adviser.

<span class="mw-page-title-main">Progress in artificial intelligence</span> How AI-related technologies evolve

Progress in artificial intelligence (AI) refers to the advances, milestones, and breakthroughs that have been achieved in the field of artificial intelligence over time. AI is a multidisciplinary branch of computer science that aims to create machines and systems capable of performing tasks that typically require human intelligence. Artificial intelligence applications have been used in a wide range of fields including medical diagnosis, economic-financial applications, robot control, law, scientific discovery, video games, and toys. However, many AI applications are not perceived as AI: "A lot of cutting edge AI has filtered into general applications, often without being called AI because once something becomes useful enough and common enough it's not labeled AI anymore." "Many thousands of AI applications are deeply embedded in the infrastructure of every industry." In the late 1990s and early 21st century, AI technology became widely used as elements of larger systems, but the field was rarely credited for these successes at the time.

DeepMind Technologies Limited, also known by its trade name Google DeepMind, is a British-American artificial intelligence research laboratory which serves as a subsidiary of Google. Founded in the UK in 2010, it was acquired by Google in 2014 and merged with Google AI's Google Brain division to become Google DeepMind in April 2023. The company is based in London, with research centres in Canada, France, Germany, and the United States.

AlphaGo is a computer program that plays the board game Go. It was developed by the London-based DeepMind Technologies, an acquired subsidiary of Google. Subsequent versions of AlphaGo became increasingly powerful, including a version that competed under the name Master. After retiring from competitive play, AlphaGo Master was succeeded by an even more powerful version known as AlphaGo Zero, which was completely self-taught without learning from human games. AlphaGo Zero was then generalized into a program known as AlphaZero, which played additional games, including chess and shogi. AlphaZero has in turn been succeeded by a program known as MuZero which learns without being taught the rules.

Darkforest is a computer go program developed by Meta Platforms, based on deep learning techniques using a convolutional neural network. Its updated version Darkfores2 combines the techniques of its predecessor with Monte Carlo tree search. The MCTS effectively takes tree search methods commonly seen in computer chess programs and randomizes them. With the update, the system is known as Darkfmcts3.

AlphaGo versus Lee Sedol, also known as the DeepMind Challenge Match, was a five-game Go match between top Go player Lee Sedol and AlphaGo, a computer Go program developed by DeepMind, played in Seoul, South Korea between 9 and 15 March 2016. AlphaGo won all but the fourth game; all games were won by resignation. The match has been compared with the historic chess match between Deep Blue and Garry Kasparov in 1997.

David Silver is a principal research scientist at Google DeepMind and a professor at University College London. He has led research on reinforcement learning with AlphaGo, AlphaZero and co-lead on AlphaStar.

Master is a version of DeepMind's Go software AlphaGo, named after the account name used online, which won 60 straight online games against human professional Go players from 29 December 2016 to 4 January 2017. This version was also used in the Future of Go Summit in May 2017. It used four TPUs on a single machine with Elo rating 4,858. DeepMind claimed that AlphaGo Master was 3-stone stronger than the version used in AlphaGo v. Lee Sedol.

The Future of Go Summit was held in May 2017 by the Chinese Go Association, Sport Bureau of Zhejiang Province and Google in Wuzhen, Zhejiang, the permanent host of the World Internet Conference. It featured five Go games involving AlphaGo and top Chinese Go players, as well as a forum on the future of AI. It was Google’s biggest public event in partnership with the Chinese government since Google China's search engine was moved out of mainland China to Hong Kong due to the government censorship in 2010. It was seen as a charm offensive launched by Google toward Chinese officials, being part of effort to reopen China's market.

AlphaGo versus Ke Jie was a three-game Go match between the computer Go program AlphaGo Master and current world No. 1 ranking player Ke Jie, being part of the Future of Go Summit in Wuzhen, China, played on 23, 25, and 27 May 2017. AlphaGo defeated Ke Jie in all three games.

AlphaGo Zero is a version of DeepMind's Go software AlphaGo. AlphaGo's team published an article in Nature in October 2017 introducing AlphaGo Zero, a version created without using data from human games, and stronger than any previous version. By playing games against itself, AlphaGo Zero: surpassed the strength of AlphaGo Lee in three days by winning 100 games to 0; reached the level of AlphaGo Master in 21 days; and exceeded all previous versions in 40 days.

Leela Zero is a free and open-source computer Go program released on 25 October 2017. It is developed by Belgian programmer Gian-Carlo Pascutto, the author of chess engine Sjeng and Go engine Leela.

<span class="mw-page-title-main">AlphaZero</span> Game-playing artificial intelligence

AlphaZero is a computer program developed by artificial intelligence research company DeepMind to master the games of chess, shogi and go. This algorithm uses an approach similar to AlphaGo Zero.

OpenAI Five is a computer program by OpenAI that plays the five-on-five video game Dota 2. Its first public appearance occurred in 2017, where it was demonstrated in a live one-on-one game against the professional player Dendi, who lost to it. The following year, the system had advanced to the point of performing as a full team of five, and began playing against and showing the capability to defeat professional teams.

Artificial intelligence and machine learning techniques are used in video games for a wide variety of applications such as non-player character (NPC) control and procedural content generation (PCG). Machine learning is a subset of artificial intelligence that uses historical data to build predictive and analytical models. This is in sharp contrast to traditional methods of artificial intelligence such as search trees and expert systems.

<span class="mw-page-title-main">MuZero</span> Game-playing artificial intelligence

MuZero is a computer program developed by artificial intelligence research company DeepMind to master games without knowing their rules. Its release in 2019 included benchmarks of its performance in go, chess, shogi, and a standard suite of Atari games. The algorithm uses an approach similar to AlphaZero. It matched AlphaZero's performance in chess and shogi, improved on its performance in Go, and improved on the state of the art in mastering a suite of 57 Atari games, a visually-complex domain.

The history of chess began nearly 1500 years ago, and over the past century and a half the game has changed drastically. No technology or strategy, however, has changed chess as much as the introduction of chess engines. Despite only coming into existence within the previous 70 years, the introduction of chess engines has molded and defined how top chess is played today.

Oriol Vinyals is a Spanish machine learning researcher at DeepMind. He is currently technical lead on Gemini, along with Noam Shazeer and Jeff Dean.

References

  1. ↑ Arulkumaran, Kai; Antoine, Cully; Togelius, Julian (July 2019). "AlphaStar: An evolutionary computation perspective". Proceedings of the Genetic and Evolutionary Computation Conference Companion. pp. 314–5. arXiv: 1902.01724 . doi:10.1145/3319619.3321894. ISBN   9781450367486. S2CID   59604439.
  2. 1 2 Russell, Stuart (8 October 2019). "Chapter 2: Intelligence in Humans and Machines". Human Compatible: Artificial Intelligence and the Problem of Control (Hardcover ed.). Viking. ISBN   978-0-525-55861-3.
  3. 1 2 Daley, Jason (30 October 2019). "A.I. Mastered Backgammon, Chess and Go. Now It Takes On StarCraft II". Smithsonian Magazine. Retrieved 26 February 2020.
  4. 1 2 Garisto, Dan (30 October 2019). "Google AI beats top human players at strategy game StarCraft II". Nature. doi:10.1038/d41586-019-03298-6. PMID   33116320. S2CID   208116679.
  5. ↑ Cheng, Jonathan (22 April 2016). "Computers That Crush Humans at Games Might Have Met Their Match: 'StarCraft'". Wall Street Journal. Retrieved 26 February 2020.
  6. ↑ Gibbs, Samuel (27 January 2014). "Google buys UK artificial intelligence startup Deepmind for £400m". The Guardian. Retrieved 26 February 2020.
  7. ↑ Byford, Sam (10 March 2016). "DeepMind founder Demis Hassabis on how AI will shape the future". The Verge. Retrieved 26 February 2020.
  8. ↑ "Google's Artificial Intelligence Project May Start Playing Video Games". Fortune. Retrieved 26 February 2020.
  9. ↑ Byford, Sam (4 November 2016). "Google DeepMind's next gaming challenge: can AI beat StarCraft II?". The Verge. Retrieved 26 February 2020.
  10. ↑ "Google DeepMind AI Declares Galactic War on StarCraft". Wired. 9 August 2017. Retrieved 26 February 2020.
  11. ↑ Lee, Timothy B. (30 January 2019). "An AI crushed two human pros at StarCraft—but it wasn't a fair fight". Ars Technica. Retrieved 26 February 2020.
  12. ↑ Amadeo, Ron (11 July 2019). "DeepMind AI is secretly lurking on the public StarCraft II 1v1 ladder". Ars Technica. Retrieved 26 February 2020.
  13. 1 2 Statt, Nick (30 October 2019). "DeepMind's StarCraft 2 AI is now better than 99.8 percent of all human players". The Verge. Retrieved 26 February 2020.
  14. 1 2 3 4 5 Kelion, Leo (30 October 2019). "DeepMind claims landmark moment for AI in esports". BBC News. Retrieved 26 February 2020.
  15. ↑ Vinyals, Oriol; Babuschkin, Igor; Czarnecki, Wojciech M.; et al. (30 October 2019). "Grandmaster level in StarCraft II using multi-agent reinforcement learning". Nature. 575 (7782): 350–354. Bibcode:2019Natur.575..350V. doi:10.1038/s41586-019-1724-z. PMID   31666705. S2CID   204972004.
  16. ↑ Heaven, Douglas (21 August 2019). "Mind meld: Artificial intelligence is improving the way humans think". New Scientist. Retrieved 26 February 2020.
  17. ↑ Lee, Alex (30 October 2019). "DeepMind has finally thrashed humans at StarCraft for real". Wired UK. Retrieved 26 February 2020.
  18. ↑ "DeepMind claims breakthrough as AI masters StarCraft II game". Financial Times . 2019. Retrieved 26 February 2020.
  19. 1 2 3 Sample, Ian (30 October 2019). "AI becomes grandmaster in 'fiendishly complex' StarCraft II". The Guardian. Retrieved 26 February 2020.
This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.