2016 Cyber Grand Challenge

Last updated
Cyber Grand Challenge (CGC)
DateAugust 4, 2016 [1]
Time9:00 am to 8:00 pm [1]
DurationEleven hours [1]
VenueParis Hotel & Conference Center [2]
Location Las Vegas, Nevada [2]

The 2016 Cyber Grand Challenge (CGC) was a challenge created by The Defense Advanced Research Projects Agency (DARPA) in order to develop automatic defense [3] systems that can discover, prove, and correct software flaws in real-time.

Contents

The event placed machine versus machine (no human intervention) in what was called the "world's first automated network defense tournament." [4]

The final event was held on August 4, 2016 at the Paris Hotel & Conference Center in Las Vegas, Nevada within the 24th DEF CON hacker convention.

It resembled in structure the long-standing "capture the flag" (CTF) security competitions, and the winning system indeed competed against humans in the "classic" DEF CON CTF held in the following days. The Cyber Grand Challenge featured, however, a more standardized scoring and vulnerability-proving system: all exploits and patched binaries were submitted and evaluated by the referee infrastructure. [5]

In addition to the CGC, DARPA has also conducted prize competitions in other areas of technology.

Background

Races develop between criminals attempting to abuse vulnerabilities and analysts who assess, remediate, check, and deploy a patch before significant damage can be done. [3] Experts adhere to a process that involves complicated reasoning followed by manual creation of each security signature and software patch, a technical process that requires months and dollars. [3] This has resulted in various software insecurities favoring attackers. [2] [3] Devices such as smart televisions, wearable technologies, and high-end home appliances that are connected to the internet aren't always produced with security in mind and moreover utility systems, power grids, and traffic lights could be more susceptible to attacks, says the DARPA. [4]

To help overcome these challenges, DARPA launched in 2014 [6] the Cyber Grand Challenge: a two-year competition seeking to create automatic defensive systems capable of reasoning about flaws, formulating patches and deploying them on a network in real time. The competition was split into two main events: an open qualification event to be held in 2015 and a final event in 2016 where only the top seven teams from the qualifiers could participate. The winner of the final event would be awarded $2 million and the opportunity to play against humans in the 24th DEF CON capture the flag competition. [7]

Technology

Challenge binaries

Challenge Binaries ran on the full 32-bit Intel x86 architecture, albeit with a simplified ABI. [8]

Reducing external interaction to its base components (e.g., system calls for well-defined I/O, dynamic memory allocation, and a single source of randomness) simplified both modeling and securely running the binaries in isolation to observe their behavior.

Internal complexity was however unrestricted, with challenges going as far as implementing a particle physics simulator, [9] chess, [10] programming/scripting languages, [11] [12] parsing of huge amounts of markup data, [13] vector graphics, [14] just-in-time compilation, [15] VMs, [16] etc.

The challenge authors were themselves scored based on how well they distinguished the players' relative performance, encouraging challenges to exercise specific weaknesses of automatic reasoning (e.g., state explosion) while remaining solvable by well-constructed systems.

Player systems

Each playing system -- a fully-automated "Cyber Reasoning System" (CRS) -- had to demonstrate ability in several areas of computer security:

Teams described their approach in various venues. [17] [18] Additionally, the third-place finisher (Shellphish) released their entire system's source code. [19]

Due to the complexity of the task, players had to combine multiple techniques and do so in a fully-unattended and time-efficient fashion. For instance, the highest attack score was reached by discovering vulnerabilities via a combination of guided fuzzing and symbolic execution -- i.e., an AFL-based fuzzer combined with the angr binary analysis framework, leveraging a QEMU-based emulation and execution-tracing system. [18]

CGC Qualification Event (CQE)

The CGC Qualification Event (CQE) was held on June 3, 2015 and lasted for 24 hours. [20] CQE had two tracks: a funded-track for seven teams selected by DARPA based on their proposals (with an award up to $750,000 per team) and an open-track where any self-funded team could participate. Over 100 teams registered internationally and 28 reached the Qualification Event. [21] During the event, teams were given 131 different programs and were challenged with finding vulnerabilities as well as fixing them automatically while maintaining performance and functionality. Collectively, all teams managed to identify vulnerabilities in 99 out of the 131 provided programs. [22] After collecting all submissions from competitors, DARPA ranked all teams based on their patching and vulnerability-finding ability.

The top seven teams and finalists in alphabetical order were: [23]

Upon qualification, each one of the above seven teams received $750,000 in funding to prepare for the final event.

CGC Final Event (CFE)

The CGC Final Event (CFE) was held on August 4, 2016 and lasted for 11 hours. [3] During the final event, finalists saw their machines face against each other in a fully automatic capture-the-flag competition. [4] Each of the seven qualifying teams competed for the top three positions that would share almost $4 million in prize money. [4]

Final results

The winning systems of the Cyber Grand Challenge (CGC) Final Event were:

  1. "Mayhem" [24] - developed by ForAllSecure, of Pittsburgh, Pa. - $2 million
  2. "Xandra" - developed by team TECHx consisting of GrammaTech Inc., Ithaca, N.Y., and UVa, Charlottesville, Va. - $1 million
  3. "Mechanical Phish" - developed by Shellphish, UC Santa Barbara, Ca. - $750,000

The other competing systems were:

See also

Related Research Articles

<span class="mw-page-title-main">DARPA</span> Technology research and development agency of the U.S. Department of Defense

The Defense Advanced Research Projects Agency (DARPA) is a research and development agency of the United States Department of Defense responsible for the development of emerging technologies for use by the military. Originally known as the Advanced Research Projects Agency (ARPA), the agency was created on February 7, 1958, by President Dwight D. Eisenhower in response to the Soviet launching of Sputnik 1 in 1957. By collaborating with academia, industry, and government partners, DARPA formulates and executes research and development projects to expand the frontiers of technology and science, often beyond immediate U.S. military requirements. The name of the organization first changed from its founding name, ARPA, to DARPA, in March 1972, changing back to ARPA in February 1993, then reverted to DARPA in March 1996.

DEF CON is a hacker convention held annually in Las Vegas, Nevada. The first DEF CON took place in June 1993 and today many attendees at DEF CON include computer security professionals, journalists, lawyers, federal government employees, security researchers, students, and hackers with a general interest in software, computer architecture, hardware modification, conference badges, and anything else that can be "hacked". The event consists of several tracks of speakers about computer and hacking-related subjects, as well as cyber-security challenges and competitions. Contests held during the event are extremely varied and can range from creating the longest Wi-Fi connection to finding the most effective way to cool a beer in the Nevada heat.

The DARPA Grand Challenge is a prize competition for American autonomous vehicles, funded by the Defense Advanced Research Projects Agency, the most prominent research organization of the United States Department of Defense. Congress has authorized DARPA to award cash prizes to further DARPA's mission to sponsor revolutionary, high-payoff research that bridges the gap between fundamental discoveries and military use. The initial DARPA Grand Challenge in 2004 was created to spur the development of technologies needed to create the first fully autonomous ground vehicles capable of completing a substantial off-road course within a limited time. The third event, the DARPA Urban Challenge in 2007, extended the initial Challenge to autonomous operation in a mock urban environment. The 2012 DARPA Robotics Challenge, focused on autonomous emergency-maintenance robots, and new Challenges are still being conceived. The DARPA Subterranean Challenge was tasked with building robotic teams to autonomously map, navigate, and search subterranean environments. Such teams could be useful in exploring hazardous areas and in search and rescue.

<span class="mw-page-title-main">Peiter Zatko</span> American computer security expert

Peiter C. Zatko, better known as Mudge, is an American network security expert, open source programmer, writer, and hacker. He is currently the chief information officer of DARPA. He was the most prominent member of the high-profile hacker think tank the L0pht as well as the computer and culture hacking cooperative the Cult of the Dead Cow.

<span class="mw-page-title-main">Fuzzing</span> Automated software testing technique

In programming and software development, fuzzing or fuzz testing is an automated software testing technique that involves providing invalid, unexpected, or random data as inputs to a computer program. The program is then monitored for exceptions such as crashes, failing built-in code assertions, or potential memory leaks. Typically, fuzzers are used to test programs that take structured inputs. This structure is specified, such as in a file format or protocol and distinguishes valid from invalid input. An effective fuzzer generates semi-valid inputs that are "valid enough" in that they are not directly rejected by the parser, but do create unexpected behaviors deeper in the program and are "invalid enough" to expose corner cases that have not been properly dealt with.

<span class="mw-page-title-main">Git</span> Distributed version control software system

Git is a distributed version control system that tracks versions of files. It is often used to control source code by programmers who are developing software collaboratively.

In software development, distributed version control is a form of version control in which the complete codebase, including its full history, is mirrored on every developer's computer. Compared to centralized version control, this enables automatic management branching and merging, speeds up most operations, improves the ability to work offline, and does not rely on a single location for backups. Git, the world's most popular version control system, is a distributed version control system.

Grand Challenges are difficult but important problems set by various institutions or professions to encourage solutions or advocate for the application of government or philanthropic funds especially in the most highly developed economies and

... energize not only the scientific and engineering community, but also students, journalists, the public, and their elected representatives, to develop a sense of the possibilities, an appreciation of the risks, and an urgent commitment to accelerate progress.

<span class="mw-page-title-main">Metasploit</span> Computer security testing tool

The Metasploit Project is a computer security project that provides information about security vulnerabilities and aids in penetration testing and IDS signature development. It is owned by Boston, Massachusetts-based security company, Rapid7.

GrammaTech is a cybersecurity research services company based in Ithaca, New York. The company was founded in 1988 as a technology spin-off of Cornell University. GrammaTech software research services include the following; software analysis, vulnerability detection and mitigation, binary transformation and hardening, and autonomous computing. In September 2023, Battery Ventures acquired GrammaTech's software products division, including the CodeSonar and CodeSentry product lines. Thus establishing a new, independent entity that will operate under the CodeSecure, Inc. name and be headquartered in Bethesda, Maryland.

Vulnerability management is the "cyclical practice of identifying, classifying, prioritizing, remediating, and mitigating" software vulnerabilities. Vulnerability management is integral to computer security and network security, and must not be confused with vulnerability assessment.

<span class="mw-page-title-main">Wargame (hacking)</span> Cyber-security challenge and mind sport in hacking

In hacking, a wargame is a cyber-security challenge and mind sport in which the competitors must exploit or defend a vulnerability in a system or application, and/or gain or prevent access to a computer system.

SCADA Strangelove is an independent group of information security researchers founded in 2012, focused on security assessment of industrial control systems (ICS) and SCADA.

<span class="mw-page-title-main">American Fuzzy Lop (software)</span> Software fuzzer that employs genetic algorithms

American Fuzzy Lop (AFL), stylized in all lowercase as american fuzzy lop, is a free software fuzzer that employs genetic algorithms in order to efficiently increase code coverage of the test cases. So far it has detected dozens of significant software bugs in major free software projects, including X.Org Server, PHP, OpenSSL, pngcrush, bash, Firefox, BIND, Qt, and SQLite.

The Information Innovation Office (I2O) is one of the seven technical offices within DARPA, an agency of the U.S. Department of Defense that is responsible for the development of advanced technology for national security. I2O was created in 2010 by combining the Information Processing Techniques Office (IPTO) and the Transformational Convergence Technology Office (TCTO). The office focuses on basic and applied research in the areas of cyber security, data analytics, and human-machine symbiosis.

<span class="mw-page-title-main">Ang Cui</span> American computer scientist

Ang Cui is an American cybersecurity researcher and entrepreneur. He is the founder and CEO of Red Balloon Security in New York City, a cybersecurity firm that develops new technologies to defend embedded systems against exploitation.

<span class="mw-page-title-main">Sakura Samurai (group)</span> Hacker group

Sakura Samurai was a white hat hacking and security research group that was founded in 2020. The group is responsible for multiple vulnerability disclosures involving governmental groups and various corporations.

<span class="mw-page-title-main">Capture the flag (cybersecurity)</span> Computer security exercise

Capture the Flag (CTF) in computer security is an exercise in which participants attempt to find text strings, called "flags", which are secretly hidden in purposefully-vulnerable programs or websites. They can be used for both competitive or educational purposes. In two main variations of CTFs, participants either steal flags from other participants or from organizers. A mixed competition combines these two styles. Competitions can include hiding flags in hardware devices, they can be both online or in-person, and can be advanced or entry-level. The game is inspired by the traditional outdoor sport of the same name. CTFs are used as a tool for developing and refining cybersecurity skills, making them popular in both professional and academic settings.

Cybersecurity in popular culture examines the various ways in which the themes and concepts related to cybersecurity have been portrayed and explored in different forms of popular culture, such as music, movies, television shows, and literature. As the digital age continues to expand and the importance of protecting computer systems, networks, and digital information grows, the awareness and understanding of cybersecurity have increasingly become a part of mainstream culture. Popular culture often portraits the gloomy underworld of cybersecurity, where unconventional tactics are used to combat a diverse range of threats to individuals, businesses, and governments. The integration of dark and mysterious elements into cybersecurity stories helps create a sense of uncertainty, rule-breaking, and intriguing ambiguity. This captures the public's attention and highlights the high stakes involved in the ongoing struggle to protect our digital world. This article highlights the creative works and cultural phenomena that have brought cybersecurity issues to the forefront, reflecting society's evolving relationship with technology, privacy, and digital security.

Over the years, the U.S. Defense Advanced Research Projects Agency (DARPA) has conducted a number of prize competitions to spur innovations. A prize competition allows DARPA to establish an ambitious goal, which makes public way for novel approaches from the public that might otherwise appear too risky to undertake by experts in a particular discipline.

References

  1. 1 2 3 "Cyber Grand Challenge Event Information for Finalists" (PDF). Cybergrandchallenge.com. Archived from the original (PDF) on 28 April 2017. Retrieved 17 July 2016.
  2. 1 2 3 "The Cyber Grand Challenge (CGC) seeks to automate cyber defense process". Cybergrandchallenge.com. Archived from the original on 1 August 2016. Retrieved 17 July 2016.
  3. 1 2 3 4 5 Walker, Michael. "a race ensues between miscreants intending to exploit the vulnerability and analysts who must assess, remediate, test, and deploy a patch before significant damage can be done". darpa.mil. Retrieved 17 July 2016.
  4. 1 2 3 4 Uyeno, Greg (5 July 2016). "Smart Televisions, wearable technologies, utility systems, power grids, and more inclined to cyber attacks". Live Science . Retrieved 17 July 2016.
  5. ↑ "CRS Team Interface API". GitHub . -- as opposed to classic CTF games, in which players directly attack each others and freely change their own VMs
  6. ↑ Chang, Kenneth (2014-06-02). "Automating Cybersecurity". The New York Times. ISSN   0362-4331 . Retrieved 2016-09-06.
  7. ↑ Tangent, The Dark. "DEF CON® 24 Hacking Conference". defcon.org. Retrieved 2016-09-06.
  8. ↑ "CGC ABI". GitHub .
  9. ↑ "CROMU_00002". GitHub .
  10. ↑ "CROMU_00005". GitHub .
  11. ↑ "KPRCA_00038". GitHub .
  12. ↑ "KPRCA_00028". GitHub .
  13. ↑ "CROMU_00015". GitHub .
  14. ↑ "CROMU_00018". GitHub .
  15. ↑ "KPRCA_00002". GitHub .
  16. ↑ "KPRCA_00014". GitHub .
  17. ↑ Dedicated special issue of the IEEE Security & Privacy journal: "Hacking Without Humans". IEEE Security & Privacy. 16 (2). IEEE Computer Society. March 2018. ISSN   1558-4046.
  18. 1 2 Publications on individual components, such as Shellphish's Stephens N, Grosen J, Salls C, Dutcher A, Wang R, Corbetta J, Shoshitaishvili Y, Kruegel C, Vigna G (2016). Driller: Augmenting Fuzzing Through Selective Symbolic Execution (PDF). Network & Distributed System Security Symposium (NDSS). Vol. 16.
  19. ↑ "Mechanical Phish". GitHub .
  20. ↑ "Cyber Grand Challenge". Archived from the original on 2016-09-11.
  21. ↑ "The DARPA Cyber Grand Challenge: A Competitor's Perspective".
  22. ↑ "Legitimate Business Syndicate: What is the Cyber Grand Challenge?". blog.legitbs.net. Retrieved 2016-09-06.
  23. ↑ "DARPA | Cyber Grand Challenge". www.cybergrandchallenge.com. Archived from the original on 2016-08-01. Retrieved 2016-09-06.
  24. 1 2 "Mayhem comes in first place at CGC". August 7, 2016. Retrieved August 13, 2016.
This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.