Openpilot

Last updated
openpilot
Original author(s) George Hotz
Developer(s) comma.ai
Initial release30 November 2016;8 years ago (2016-11-30)
Stable release
0.9.7 (13 June 2024;6 months ago (2024-06-13))
Repository github.com/commaai/openpilot
Written in Python, C++, C
Operating system AGNOS, based on Ubuntu [1]
Platform Qualcomm Snapdragon 845 [2]
Standard(s) ISO 26262, MISRA C:2012 [3]
Available in10 languages [4] [5] [6]
List of languages
Arabic, Chinese (Simplified), Chinese (Traditional), French, German, Japanese, Korean, Portuguese, Thai, Turkish
Type Computer vision
License MIT License
Website comma.ai
Toyota Prius with openpilot installed Commaai car Feb 2019.png
Toyota Prius with openpilot installed
An old version of Cabana, a CAN Bus traffic visualiser, now part of openpilot tools Commaai cabana screenshot Feb 2019.png
An old version of Cabana, a CAN Bus traffic visualiser, now part of openpilot tools
A user annotating a drive Comma Explorer Drive Annotation Tool.png
A user annotating a drive

openpilot is an open-source, semi-automated driving software by comma.ai, Inc. When paired with comma hardware, it replaces advanced driver-assistance systems in various cars, improving over the original system. [7] [8] As of 2023, openpilot supports 250+ car models and has 6000+ users, accumulating over 90 million miles (140,000,000 km). [8] [9] [10] [11]

Contents

openpilot runs on comma 2/3/X hardware, also developed by comma.ai. Packaged as an aftermarket retrofit, it allows users to enhance their existing cars with upgraded computing power, enhanced vision, and regularly updated software. openpilot has been cited to offer a "natural" and human-like driving experience, [8] and reviewed favorably for its ease of use and driver engagement. [7] As of May 2024, openpilot holds the record for the shortest time in a semi-autonomous, coast-to-coast drive across the U.S. [12] [13]

History

comma.ai Inc was founded in September 2015 by George Hotz. [14] The first version of openpilot was revealed a few months later in a Bloomberg article, showing functionality on a 2016 Acura ILX. [15] The video and article instigated a cease and desist letter from the California Department of Motor Vehicles, claiming comma.ai was testing a self-driving car without a license.

openpilot was packaged into a small, shippable device called the "comma one", announced at TechCrunch Disrupt. [16] On October 27, 2016, NHTSA issued a Special Order to comma.ai demanding detailed information about the comma one, to determine if the device complies with legally required Federal Motor Vehicle Safety Standards. [17] comma.ai responded in a tweet from Shenzhen, China, announcing the cancellation of the comma one. [18]

comma.ai open-sourced openpilot a month later, on November 30, 2016, emphasizing its intended use for research, without any warranty. [19] [20]

On January 7, 2020, comma.ai introduced the $999 "comma two" device at CES in Las Vegas. [21] [22] [23] In November 2020, Consumer Reports ranked openpilot above all other advanced driver-assistance systems on the market, including Tesla's Autopilot, Cadillac's Super Cruise and Ford's Co-pilot 360. In particular, openpilot's performance in the areas of driver engagement and ease of use scored better than all other systems evaluated by Consumer Reports. [7]

comma two running the openpilot software Comma Two Running Openpilot.jpg
comma two running the openpilot software

At comma's 2021 comma_con convention, the company released the "comma three devkit". [24] This model uses two fish eye cameras oriented to the front and a long distance camera oriented to the rear to analyze the car's surroundings. [25]

In January 2023, The Verge found openpilot-based driving-assist technology to be better than that from legacy car manufacturers, delivering more "natural" and human-like driving experience. Nearly 5000 units of openpilot-supported Comma 3 devices were already said to be distributed. [8]

At comma's 2023 COMMA_CON convention, the "comma 3X" was announced as a successor to the comma three devkit at a lower price of $1249. [26] In 2023, the total distance driven by openpilot's 6000+ users was said to have exceeded 90 million miles, [10] [11] an improvement over the 25 million miles figure reported in 2020. [27]

In April 2024, a new semi-autonomous Cannonball run record, involving a drive across the United States from the East Coast to the West Coast, was set using a 2017 Toyota Prius equipped with a comma 3X unit running openpilot. The drive took 43 hours and 18 minutes at 98.4% autonomy, beating the previous record set in a Tesla Model S (97.7% autonomy) by nearly 12 hours. [12] [13] [28]

Comma Hardware

ModelRelease DateRelease Price
comma one13 Sept 2016$999 [29]
EON Dashcam DevKit19 Oct 2017$699 [30]
comma two devkit7 Jan 2020$999 [31]
comma three devkit31 July 2021$2199 [32]
comma 3X30 July 2023$1250 [33]

Features

In contrast to traditional autonomous driving solutions where the perception, prediction, and planning units are separate "modules", openpilot adopts a system-level end-to-end design to predict the car's trajectory directly from the camera images. openpilot's end-to-end design is a neural network that is trained by comma.ai using real-world driving data uploaded by openpilot users. [34]

openpilot 0.9.4 and onwards include "navigate on openpilot", a feature akin to "Full Self-Driving" (FSD) from Tesla, Inc. [35]

Automated lane-centering

openpilot uses machine learning, trained with real-world driving data, to determine the safest path on the road. This improves perception on roads without lane markings vs. lane-centering by tracking current lane lines.

Adaptive cruise control

openpilot maintains a safe following distance from the vehicle ahead. It is capable of driving in stop-and-go traffic with no user intervention. It uses OpenStreetMap's road curvature and speed limit data to allow slowing on sharp turns and setting the vehicle's desired speed to the current speed limit on certain community-maintained forks of the main repository.

Driver monitoring

openpilot recognizes the driver's face; if the driver is distracted, openpilot warns the driver. If the driver is distracted for more than six seconds, openpilot decelerates the vehicle to a stop, and audibly alerts the user.

Assisted lane change

openpilot uses the model to change lanes when the user engages the turn signal: a nudge is optional on the steering wheel to confirm the lane change. openpilot also interacts with the blind spot monitor on certain makes and models to block the lane change in the event a car is detected by the blind spot monitor.

Software updates

openpilot receives over-the-air software updates via WiFi or a cellular phone network, (OTA updates).

Supported cars

At launch, openpilot supported the Honda Civic and the Acura ILX. Support for more vehicles and brands has been added through open-source contribution. [36] As of November 2023, the list of officially supported vehicles contains more than 250 entries, [9] and can be viewed at https://comma.ai/vehicles.

Community

Development is supported by an open-source community using Discord [37] and GitHub.

comma.ai has released tools and guides to help developers port their cars. [38] In addition, they released tools to let users review their drives. [39]

Forks

comma.ai maintains the openpilot codebase and releases, and there is a growing community that maintains various forks of openpilot. These forks consist of experimental features such as stop light detection.

Pre-Autopilot Tesla models have been retrofitted with openpilot through a community fork. [40] Chrysler and Jeep models have also gained support through community contributions. [41] [42]

There are over 6,600 forks of the openpilot GitHub repository. [20]

Related Research Articles

<span class="mw-page-title-main">Self-driving car</span> Vehicle operated with reduced human input

A self-driving car, also known as a autonomous car (AC), driverless car, robotaxi, robotic car or robo-car, is a car that is capable of operating with reduced or no human input. Self-driving cars are responsible for all driving activities, such as perceiving the environment, monitoring important systems, and controlling the vehicle, which includes navigating from origin to destination.

<span class="mw-page-title-main">Advanced driver-assistance system</span> Electronic systems that help a vehicle driver while driving or parking

Advanced driver-assistance systems (ADAS) are technologies that assist drivers with the safe operation of a vehicle. Through a human-machine interface, ADAS increase car and road safety. ADAS use automated technology, such as sensors and cameras, to detect nearby obstacles or driver errors, and respond accordingly. ADAS can enable various levels of autonomous driving.

<span class="mw-page-title-main">Lane departure warning system</span> Mechanism designed to warn a driver when the vehicle begins to move out of its lane

In road-transport terminology, a lane departure warning system (LDWS) is a mechanism designed to warn the driver when the vehicle begins to move out of its lane on freeways and arterial roads. These systems are designed to minimize accidents by addressing the main causes of collisions: driver error, distractions and drowsiness. In 2009 the U.S. National Highway Traffic Safety Administration (NHTSA) began studying whether to mandate lane departure warning systems and frontal collision warning systems on automobiles.

<span class="mw-page-title-main">Teleoperation</span> Operation of a system or machine at a distance

Teleoperation indicates operation of a system or machine at a distance. It is similar in meaning to the phrase "remote control" but is usually encountered in research, academia and technology. It is most commonly associated with robotics and mobile robots but can be applied to a whole range of circumstances in which a device or machine is operated by a person from a distance.

<span class="mw-page-title-main">Vehicular automation</span> Automation for various purposes of vehicles

Vehicular automation is the use of technology to assist or replace the operator of a vehicle such as a car, truck, aircraft, rocket, military vehicle, or boat. Assisted vehicles are semi-autonomous, whereas vehicles that can travel without a human operator are autonomous. The degree of autonomy may be subject to various constraints such as conditions. Autonomy is enabled by advanced driver-assistance systems (ADAS) of varying capacity.

<span class="mw-page-title-main">Automatic parking</span> Autonomous car-maneuvering system

Automatic parking is an autonomous car-maneuvering system that moves a vehicle from a traffic lane into a parking spot to perform parallel, perpendicular, or angle parking. The automatic parking system aims to enhance the comfort and safety of driving in constrained environments where much attention and experience is required to steer the car. The parking maneuver is achieved by means of coordinated control of the steering angle and speed which takes into account the actual situation in the environment to ensure collision-free motion within the available space.

<span class="mw-page-title-main">George Hotz</span> American software engineer

George Francis Hotz, alias geohot, is an American security hacker, entrepreneur, and software engineer. He is known for developing iOS jailbreaks, reverse engineering the PlayStation 3, and for the subsequent lawsuit brought against him by Sony. From September 2015 onwards, he has been working on his vehicle automation machine learning company comma.ai. Since November 2022, Hotz has been working on tinygrad, a deep learning framework.

<span class="mw-page-title-main">Alex Roy</span> American racing driver

Alexander Roy is an American writer, podcaster, TV host and rally race driver who has set various endurance driving records, including the US "Cannonball Run" transcontinental driving record, which he and Dave Maher broke in 2007 in 31 hours and 4 minutes, featured in the 2019 documentary APEX: The Secret Race Across America.

A connected car is a car that can communicate bidirectionally with other systems outside of the car. This connectivity can be used to provide services to passengers or to support or enhance self-driving functionality. For safety-critical applications, it is anticipated that cars will also be connected using dedicated short-range communications (DSRC) or cellular radios, operating in the FCC-granted 5.9 GHz band with very low latency.

<span class="mw-page-title-main">Mobileye</span> Israeli information technology company

Mobileye Global Inc. is a United States-domiciled, Israel-headquartered autonomous driving company. It is developing self-driving technologies and advanced driver-assistance systems (ADAS) including cameras, computer chips, and software. Mobileye was acquired by Intel in 2017 and went public again in 2022.

<span class="mw-page-title-main">History of self-driving cars</span> Overview of the history of self-driving cars

Experiments have been conducted on self-driving cars since 1939; promising trials took place in the 1950s and work has proceeded since then. The first self-sufficient and truly autonomous cars appeared in the 1980s, with Carnegie Mellon University's Navlab and ALV projects in 1984 and Mercedes-Benz and Bundeswehr University Munich's Eureka Prometheus Project in 1987. In 1988, William L Kelley patented the first modern collision Predicting and Avoidance devices for Moving Vehicles. Then, numerous major companies and research organizations have developed working autonomous vehicles including Mercedes-Benz, General Motors, Continental Automotive Systems, Autoliv Inc., Bosch, Nissan, Toyota, Audi, Volvo, Vislab from University of Parma, Oxford University and Google. In July 2013, Vislab demonstrated BRAiVE, a vehicle that moved autonomously on a mixed traffic route open to public traffic.

<span class="mw-page-title-main">Robotaxi</span> Taxi without a human driver

A robotaxi, also known as robot taxi, robo-taxi, self-driving taxi or driverless taxi, is an autonomous car operated for a ridesharing company.

Nvidia Drive is a computer platform by Nvidia, aimed at providing autonomous car and driver assistance functionality powered by deep learning. The platform was introduced at the Consumer Electronics Show (CES) in Las Vegas in January 2015. An enhanced version, the Drive PX 2 was introduced at CES a year later, in January 2016.

<span class="mw-page-title-main">Tesla Autopilot</span> Suite of advanced driver-assistance system features by Tesla

Tesla Autopilot is an advanced driver-assistance system (ADAS) developed by Tesla that amounts to partial vehicle automation. Tesla provides "Base Autopilot" on all vehicles, which includes Autosteer, and traffic-aware cruise control. Owners may purchase or subscribe to Full Self-Driving (FSD) which adds semi-autonomous navigation that responds to traffic lights and stop signs, lane change assistance, self-parking, and the ability to summon the car from a garage or parking spot.

Increases in the use of autonomous car technologies are causing incremental shifts in the control of driving. Liability for incidents involving self-driving cars is a developing area of law and policy that will determine who is liable when a car causes physical damage to persons or property. As autonomous cars shift the control of driving from humans to autonomous car technology, there is a need for existing liability laws to evolve to reasonably identify the appropriate remedies for damage and injury. As higher levels of autonomy are commercially introduced, the insurance industry may see higher proportions of commercial and product liability lines of business, while the personal automobile insurance line of business shrinks.

kar-go Autonomous delivery vehicle

Kar-go, is an autonomous delivery vehicle, designed and built by British company, Academy of Robotics Ltd, a UK company, registered in Wales. The vehicle uses self-drive / driverless car technology to drive itself to locations where it delivers packages autonomously.

<span class="mw-page-title-main">Lane centering</span> Mechanism designed to keep a car centered in the lane

In road-transport terminology, lane centering, also known as lane centering assist, lane assist, auto steer or autosteer, is an advanced driver-assistance system that keeps a road vehicle centered in the lane, relieving the driver of the task of steering. Lane centering is similar to lane departure warning and lane keeping assist, but rather than warn the driver, or bouncing the car away from the lane edge, it keeps the car centered in the lane. Together with adaptive cruise control (ACC), this feature may allow unassisted driving for some length of time. It is also part of automated lane keeping systems.

<span class="mw-page-title-main">Cannonball Run challenge</span> Unsanctioned vehicle speed record

A Cannonball Run is an unsanctioned speed record for driving across the United States, typically accepted to run from New York City's Red Ball Garage to the Portofino Hotel in Redondo Beach near Los Angeles, covering a distance of about 2,906 miles (4,677 km). As of October 2021, the overall record is 25 hours 39 minutes, with an average speed of 112 miles per hour (180 km/h), driven by Arne Toman and Doug Tabbutt.

The Sony Vision-S is an all-electric automobile platform that was developed by Sony, with the first model unveiled at the 2020 Consumer Electronics Show (CES) in Las Vegas as the Vision-S Concept. When the second concept was unveiled in 2022 as the Vision-S 02, the original concept was retroactively renamed to Vision-S 01. Sony plans to use the Vision-S platform to develop future electric vehicles under a new wholly-owned subsidiary, Sony Mobility Inc., which will assume responsibility for the Vision-S platform and vehicle development in the spring of 2022. The Vision-S platform is fully electric and designed to emphasize the user experience of autonomous driving, as well as accommodate the growth of technology in future models.

Tesla Autopilot, an advanced driver-assistance system for Tesla vehicles, uses a suite of sensors and an onboard computer. It has undergone several hardware changes and versions since 2014, most notably moving to an all-camera-based system by 2023, in contrast with ADAS from other companies, which include radar and sometimes lidar sensors.

References

  1. ↑ "openpilot in 2021". 12 October 2021.
  2. ↑ "Comma.ai announces Comma Three hardware with OLED display and 360° view of the car". 3 August 2021.
  3. ↑ "Github - openpilot safety". GitHub .
  4. ↑ "Github - openpilot language support". GitHub .
  5. ↑ "openpilot 0.9.5". Comma.ai Blog. 17 November 2023.
  6. ↑ "openpilot 0.8.16". 26 August 2022.
  7. 1 2 3 Stumpf, Rob (2020-11-25). "Consumer Reports Ranks This Aftermarket Driver Assistance Kit Above Tesla Autopilot, Cadillac Super Cruise". The Drive. Retrieved 2023-11-20.
  8. 1 2 3 4 Hall, Emme (2023-01-10). "George Hotz wants to "make driving chill" with next-gen Comma 3 hands-free driver assist". The Verge. Retrieved 2023-11-20.
  9. 1 2 Leedy, Savannah (2023-10-27). "New Autonomous Device Available to Consumers Offers Affordable Self-Driving Experience". Arizona Auto News - Events - Reviews. Retrieved 2023-11-21.
  10. 1 2 "Robotaxi, Robotrucks, RoboFactories and RoboEverything | NextBigFuture.com". 2023-07-21. Retrieved 2023-11-20.
  11. 1 2 "openpilot - open source advanced driver assistance system". comma.ai. Retrieved 2023-11-21.
  12. 1 2 Dorr, Bryon (2024-05-08). "Prius Sets 'Autonomous' Cannonball Run Record With AI Driving Assistant". Yahoo Tech. Retrieved 2024-05-14.
  13. 1 2 Roy, Alex (2016-09-21). "Alex Roy Shatters Electric and Autonomous Cannonball Run Records in a Tesla". The Drive. Retrieved 2024-05-21.
  14. ↑ "Comma.ai founder George Hotz wants to free humanity from the AI simulation". 2019-03-09. Retrieved 2020-03-10.
  15. ↑ "The First Person to Hack the iPhone Built a Self-Driving Car. In His Garage:George Hotz is taking on Google and Tesla by himself". Bloomberg.com. Archived from the original on 2016-02-01.
  16. ↑ "George "Geohot" Hotz Presents the Comma One at Disrupt SF". YouTube. 2016-09-13. Retrieved 2020-03-02.
  17. ↑ "NHTSA Special Order".
  18. ↑ "The comma one is cancelled. comma.ai will be exploring other products and markets. Hello from Shenzhen, China. -GH 3/3". October 28, 2016.
  19. ↑ "George Hotz is giving away the code behind his self-driving car project". 30 November 2016. Retrieved May 13, 2017.
  20. 1 2 "commaai/openpilot". March 12, 2022 – via GitHub.
  21. ↑ Hall, Emme (2020-01-09). "We hit the road with Comma.ai's assisted-driving tech at CES 2020". Roadshow. Retrieved 2020-01-19.
  22. ↑ Baldwin, Roberto (2020-01-13). "Comma Ai continues to impress with its aftermarket driver assistance tech". Engadget. Retrieved 2020-01-19.
  23. ↑ "Available for purchase now". January 7, 2020.
  24. ↑ "comma three devkit". comma shop. comma.ai. Retrieved 4 August 2022.
  25. ↑ Archived at Ghostarchive and the Wayback Machine : COMMA_CON | July 31, 2021 | San Diego. YouTube .
  26. ↑ Hotz, George (2023-07-30). "Have you ordered a comma 3X yet?". Twitter. Retrieved 2023-07-30.
  27. ↑ Stumpf, Rob (2020-10-15). "We Tested OpenPilot, the $1,199 Device That Adds Entry-Level Autonomy to Your Car". The Drive. Retrieved 2023-11-20.
  28. ↑ Hawkins, Andrew J. (2017-07-09). "Two guys did a coast-to-coast 'Cannonball Run' in a Tesla Model S for a new record". The Verge. Retrieved 2024-05-21.
  29. ↑ TechCrunch (2016-09-13). George "Geohot" Hotz Presents the Comma One at Disrupt SF . Retrieved 2025-01-08– via YouTube.
  30. ↑ "EON Dashcam DevKit – comma.ai shop". web.archive.org. 2017-10-20. Retrieved 2025-01-08.
  31. ↑ "comma two devkit | comma.ai shop". web.archive.org. 2020-01-13. Retrieved 2025-01-08.
  32. ↑ comma (2021-08-04). George Hotz Announcing COMMA THREE at COMMA_CON . Retrieved 2025-01-08– via YouTube.
  33. ↑ comma (2023-08-04). COMMA_CON 2023 | George Announces comma 3X Hardware | Three Stories - The Past, Future, and Present . Retrieved 2025-01-08– via YouTube.
  34. ↑ Chen, Li; Tang, Tutian; Cai, Zhitian; Li, Yang (2022-06-16). "Level 2 Autonomous Driving on a Single Device: Diving into the Devils of Openpilot". arXiv: 2206.08176 [cs.CV].
  35. ↑ Agatie, Cristian (2023-07-31). "Geohot's Comma AI Launches Comma 3X ADAS Hardware With Support for Over 250 Car Models". autoevolution. Retrieved 2023-11-20.
  36. ↑ "commaai/openpilot". GitHub. 5 October 2021.
  37. ↑ "comma ai on Twitter: "Moving from Slack to Discord"". Twitter.com. Comma.ai, Inc. Retrieved 27 February 2019.
  38. ↑ ai, comma (August 4, 2018). "How to write a car port for openpilot". Medium.
  39. ↑ ai, comma (July 13, 2018). "The Half Way Point". Medium.
  40. ↑ "Tesla - comma wiki". community.comma.ai.
  41. ↑ "Giraffe for FCA group released! openpilot now officially supports only the Chrysler Pacifica Hybrid, but it will soon spread to all Chrysler, Jeep and Fiat with ACC and LaneSense!https://comma.ai/shop/products/giraffe#&gid=1&pid=9 …". February 1, 2019.
  42. ↑ "adhintz/openpilot". GitHub. 21 April 2021.
This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.