Face ID

Last updated
Face ID
Developer(s) Apple Inc.
Initial releaseNovember 2017;6 years ago (2017-11)
Operating system
License Proprietary license
Website support.apple.com/en-us/HT208108   OOjs UI icon edit-ltr-progressive.svg

Face ID is a facial recognition system designed and developed by Apple Inc. for the iPhone and iPad Pro. The system allows biometric authentication for unlocking a device, [1] making payments, accessing sensitive data, providing detailed facial expression tracking for Animoji, as well as six degrees of freedom (6DOF) head-tracking, eye-tracking, and other features. Initially released in November 2017 with the iPhone X, it has since been updated and introduced to several new iPhone models, and Many iPad Pro models. (2018 iPad Pros and later) [2]

Contents

The Face ID hardware consists of a sensor with three modules; a laser [3] dot projector that projects a grid of small infrared dots onto a user's face, a module called the flood illuminator that shines infrared light at the face, and an infrared camera which takes an infrared picture of the user, reads the resulting pattern and generates a 3D facial map. This map is compared with the registered face using a secure subsystem, and the user is authenticated if the two faces match sufficiently. The system can recognize faces with glasses, clothing, makeup, and facial hair, and adapts to changes in appearance over time.

Face ID has sparked a number of debates about security and privacy. Apple claims that Face ID is statistically more advanced than Touch ID fingerprint scanning. [4] It exhibits significantly fewer false positives. Multiple security features largely limit the risk of the system being bypassed using photos or masks, and only one proof-of-concept attempt using detailed scans has succeeded. Debate continues over the lack of legal protections offered by biometric systems as compared to passcode authentication in the United States. Privacy advocates have also expressed concern about third-party app developers' access to "rough maps" of user facial data, despite rigid requirements by Apple of how developers handle facial data.

With the onset of the COVID-19 pandemic, it was noted that Face ID was unable to recognize users wearing face coverings on some devices. [5] [6] Apple responded to criticism by offering faster fallback to passcode input, and the option for Apple Watch users to confirm whether they intended to unlock their iPhone. [7] In March 2022, Apple released iOS 15.4 which adds mask-compatible Face ID for iPhone 12 and later devices. [8]

History

Apple announced Face ID during the unveiling of the iPhone X on September 12, 2017. [9] The system was presented as the successor to Touch ID, Apple's previous fingerprint-based authentication technology embedded in the home button of the iPhone 8 and earlier devices in addition to the second and third-generation iPhone SE. [10] On September 12, 2018, Apple introduced the iPhone XS and XR with faster neural network processing speeds, providing a significant speed increase to Face ID. On October 30, 2018, Apple introduced the third generation iPad Pro, which brings Face ID to the iPad and allows face recognition in any orientation. [11] iOS 13 included an upgraded version of Face ID which is up to 30% faster than Face ID on previous versions. [12]

Technology

Infrared dots projected by an iPhone with Face ID Apple Face ID infrared dot projector.jpg
Infrared dots projected by an iPhone with Face ID
Face ID prompt before viewing the Recently Deleted album, on an iPhone 11 running iOS 16. Face ID prompt.png
Face ID prompt before viewing the Recently Deleted album, on an iPhone 11 running iOS 16.

Face ID's technology is based on PrimeSense's previous work with low-cost infrared depth perception that was the basis of the Kinect motion sensor for the Xbox console line from Microsoft; Apple had acquired PrimeSense in 2013 after Microsoft started to wane on the use of Kinect. [13] [14]

Face ID is based on a facial recognition sensor that consists of two parts: a vertical-cavity surface-emitting laser dot projector module that projects more than 30,000 infrared dots onto the user's face, and an infrared camera module that reads the pattern. [15] The pattern is projected from the laser using an Active Diffractive Optical Element which divides the beam into 30,000 dots. [16]

The pattern is encrypted and sent to a local "Secure Enclave" in the device's CPU to confirm a match with the registered face. [17] [18] The stored facial data is a mathematical representation of key details of the face, and it is inaccessible to Apple or other parties. [17] To avoid involuntary authentication, the system requires the user to open their eyes and look at the device to attempt a match, although this can be disabled through an accessibility setting. [17] Face ID is temporarily disabled and the user's passcode is required after 5 unsuccessful scans, 48 hours of inactivity, restarting the device, or if both of the device's side buttons are held briefly. [19]

Apple claimed the probability of someone else unlocking a phone with Face ID is 1 in 1,000,000 as opposed to Touch ID at 1 in 50,000. [10] [20] During initial setup, the user's face is scanned twice from a number of angles to create a complete reference map. As the system is used, it learns about typical variations in a user's appearance, and will adjust its registered face data to match aging, facial hair growth, and other changes using the Neural Engine. The system will recognize a face wearing hats, scarves, glasses, most sunglasses, [21] facial hair or makeup. [22] It also works in the dark by invisibly illuminating the whole face with a dedicated infrared flash module. [23]

Authentication with Face ID is used to enable a number of iOS features, including unlocking the phone automatically on wake, making payments with Apple Pay, and viewing saved passwords. Apps by Apple or third-party developers can protect sensitive data with a system framework; the device will verify the user's identity and return success or failure without sharing face data with the app. Additionally, Face ID can be used without authentication to track over 50 aspects of a user's facial expression and positioning, which can be used to create live effects such as Animoji or camera filters. In recent years, third-party developers have developed more use cases for FaceID such as e.g. Eyeware Beam, an iOS app that provides a reliable and precise, multi-purpose head and eye-tracking tool. It is used to enable control of the camera angle through head-motion-in games and eye-tracking to share attention with audience in streams, but also augmentative and alternative communication (AAC) and biometric research. [24]

Devices with Face ID

Safety

Face ID uses an infrared flood illuminator and laser infrared dot projector, though Apple insists that the output is low enough that it will cause no harm to the eyes or skin, and meets 'international safety standards'. They do not, however, recommend the sensor be repaired by third parties, citing security concerns. There is also an inbuilt feature to deactivate Face ID should unauthorized components be found. [25]

Issues

Twins and close relatives

Inconsistent results have been shown when testing Face ID on identical twins, with some tests showing the system managing to separate the two, [26] while other tests have failed. [27] The system has additionally been fooled by close relatives. [28] Apple states that the probability of a false match is different for twins and siblings, as well as children under 13 years of age, as "their distinct facial features may not have fully developed". [29]

Law enforcement access

Face ID has raised concerns regarding the possibility of law enforcement accessing an individual's phone by pointing the device at the user's face. [30] United States Senator Al Franken asked Apple to provide more information on the security and privacy of Face ID a day after the announcement, [31] with Apple responding by highlighting the recent publication of a security white paper and knowledge base detailing answers. [32] [33]

The Verge noted that courts in the United States have granted different Fifth Amendment rights to keycode and biometric unlocking systems. Keycodes are considered "testimonial" evidence based on the contents of users' thoughts, whereas fingerprints are considered physical evidence, with some suspects having been ordered to unlock their phones via fingerprint. [34]

In August 2018, the FBI obtained a warrant to search the property (which includes electronic devices) of a man accused of transmitting child pornography; they unlocked the suspect's iPhone by holding it up to his face, without needing his passcode. [35]

Infiltration

Many people have attempted to fool Face ID with sophisticated masks, though most have failed. [36] In November 2017, Vietnamese security firm Bkav announced in a blog post that it had created a $150 mask that successfully unlocked Face ID, but WIRED noted that Bkav's technique was more of a "proof-of-concept" rather than active exploitation risk, with the technique requiring a detailed measurement or digital scan of the iPhone owner's face, putting the real risk of danger only to targets of espionage and world leaders. [37] [38]

Third-party developers

If the user explicitly grants a third-party app permission to use the camera, the app can also access basic facial expression and positioning data from Face ID for features such as precise selfie filters such as those seen in Snapchat, or game characters mirroring real-world user facial expressions. The data accessible to third parties is not sufficient to unlock a device or even identify a user, and Apple prohibits developers from selling the data to others, creating profiles on users, or using the data for advertising. The American Civil Liberties Union and the Center for Democracy and Technology raised privacy questions about Apple's enforcement of the privacy restrictions connected to third-party access, with Apple maintaining that its App Store review processes were effective safeguards. Jay Stanley, a senior policy analyst with the ACLU, has stated that the overall idea of letting developers access sensitive facial information was still not satisfactorily handled, with Stanley telling Reuters that "the privacy issues around of the use of very sophisticated facial recognition technology for unlocking the phone have been overblown. The real privacy issues have to do with the access by third-party developers". [39] [40]

Use with face masks

During the COVID-19 pandemic, face masks were employed as a public and personal health control measure against the spread of SARS-CoV-2. Face ID at the time was incompatible with face masks, with Apple stating "Face ID is designed to work with your eyes, nose and mouth visible." [41] With the release of iOS 13.5, Apple added a feature that automatically brought up the passcode screen if it detected that the user was wearing a mask. [42] [43] Apple was criticized for not addressing these issues with the release of the iPhone 12, but was praised for the lack of inclusion of Face ID in favor of Touch ID integration into the power button on the fourth-generation iPad Air. [5] [44] In April 2021, Apple released iOS 14.5 and watchOS 7.4 with an option to allow Apple Watch to act as a backup if Face ID fails due to face masks. [7] In March 2022, Apple released iOS 15.4 which adds mask-compatible Face ID for iPhone 12 and later devices. [8]

See also

Related Research Articles

iPhone Line of smartphones by Apple Inc.

The iPhone is a line of smartphones produced by Apple that use Apple's own iOS mobile operating system. The first-generation iPhone was announced by then–Apple CEO Steve Jobs on January 9, 2007. Since then, Apple has annually released new iPhone models and iOS updates. As of November 1, 2018, more than 2.2 billion iPhones had been sold.

iOS Mobile operating system by Apple

iOS is a mobile operating system developed by Apple Inc. exclusively for its smartphones. It was unveiled in January 2007 for the first-generation iPhone, launched in June 2007.

<span class="mw-page-title-main">Siri</span> Software-based personal assistant from Apple Inc.

Siri is the digital assistant that is part of Apple Inc.'s iOS, iPadOS, watchOS, macOS, tvOS, audioOS, and visionOS operating systems. It uses voice queries, gesture based control, focus-tracking and a natural-language user interface to answer questions, make recommendations, and perform actions by delegating requests to a set of Internet services. With continued use, it adapts to users' individual language usages, searches, and preferences, returning individualized results.

Apple ID is a user account by Apple for their devices and software. Apple IDs contain the user's personal data and settings. When an Apple ID is used to log in to an Apple device, the device will automatically use the data and settings associated with the Apple ID.

iOS 6 2012 mobile operating system

iOS 6 is the sixth major release of the iOS mobile operating system developed by Apple Inc, being the successor to iOS 5. It was announced at the company's Worldwide Developers Conference on June 11, 2012, and was released on September 19, 2012. It was succeeded by iOS 7 on September 18, 2013.

<span class="mw-page-title-main">Apple Wallet</span> Digital wallet platform by Apple

Apple Wallet is a digital wallet developed by Apple Inc. and included with iOS and watchOS that allows users to store Wallet passes such as coupons, boarding passes, student ID cards, government ID cards, business credentials, resort passes, car keys, home keys, event tickets, public transportation passes, store cards, and – starting with iOS 8.1 – credit cards, and debit cards for use via Apple Pay.

A lock screen is a computer user interface element used by various operating systems. They regulate immediate access to a device by requiring the user to perform a certain action in order to receive access, such as entering a password, using a certain button combination, or performing a certain gesture using a device's touchscreen. There are various authentication methods to get past the lock screen, with the most popular and common ones being personal identification numbers (PINs), the Android pattern lock, and biometrics.

iOS 7 2013 mobile operating system

iOS 7 is the seventh major release of the iOS mobile operating system developed by Apple Inc., being the successor to iOS 6. It was announced at the company's Worldwide Developers Conference on June 10, 2013, and was released on September 18 of that year. It was succeeded by iOS 8 on September 17, 2014.

<span class="mw-page-title-main">Touch ID</span> Electronic fingerprint recognition feature by Apple

Touch ID is an electronic fingerprint recognition feature designed and released by Apple Inc. that allows users to unlock devices, make purchases in the various Apple digital media stores, and authenticate Apple Pay online or in apps. It can also be used to lock and unlock password-protected notes on iPhone and iPad. Touch ID was first introduced in iPhones with the iPhone 5s in 2013. In 2015, Apple introduced a faster second-generation Touch ID in the iPhone 6s; a year later in 2016, it made its laptop debut in the MacBook Pro integrated on the right side of the Touch Bar. Touch ID has been used on all iPads since the iPad Air 2 was introduced in 2014. In MacBooks, each user account can have up to three fingerprints, and a total of five fingerprints across the system. Fingerprint information is stored locally in a secure enclave on the Apple A7 and later chips, not in the cloud, a design choice intended to secure fingerprint information from users or malicious attackers.

<span class="mw-page-title-main">HomeKit</span> Software framework by Apple for home automation

HomeKit, also known as Apple Home, is a software framework and communication protocol developed by Apple Inc. that lets users configure, communicate with and control smart-home appliances using Apple devices. It provides users with a way to automatically discover such devices and configure them. By designing rooms, items and actions in HomeKit, users can enable automations in the home through a voice command to Siri or through the Home app. With HomeKit, developers are able to create complex applications in order to manage accessories at a high level.

iOS 9 2015 mobile operating system

iOS 9 is the ninth major release of the iOS mobile operating system developed by Apple Inc., being the successor to iOS 8. It was announced at the company's Worldwide Developers Conference on June 8, 2015, and was released on September 16, 2015. It was succeeded by iOS 10 on September 13, 2016.

iOS 10 2016 mobile operating system

iOS 10 is the tenth major release of the iOS mobile operating system developed by Apple Inc., being the successor to iOS 9. It was announced at the company's Worldwide Developers Conference on June 13, 2016, and was released on September 13, that year. It was succeeded by iOS 11 on September 19, 2017.

iOS 11 2017 mobile operating system

iOS 11 is the eleventh major release of the iOS mobile operating system developed by Apple Inc., being the successor to iOS 10. It was announced at the company's Worldwide Developers Conference on June 5, 2017, and was released on September 19, 2017. It was succeeded by iOS 12 on September 17, 2018.

iPhone X 2017 eleventh-generation smartphone designed by Apple Inc.

The iPhone X is a smartphone designed, developed and marketed by Apple. It is part of the 11th generation of the iPhone. Available for pre-order from October 27, 2017, it was released on November 3, 2017. The naming of the iPhone X is to mark the 10th anniversary of the iPhone.

<span class="mw-page-title-main">Google Pay (payment method)</span> Mobile payments platform developed by Google

Google Pay is a mobile payment service developed by Google to power in-app, online, and in-person contactless purchases on mobile devices, enabling users to make payments with Android phones, tablets, or watches. Users can authenticate via a PIN, passcode, or biometrics such as 3D face scanning or fingerprint recognition.

iOS 12 2018 mobile operating system

iOS 12 is the twelfth major release of the iOS mobile operating system developed by Apple Inc. Aesthetically similar to its predecessor, iOS 11, it focuses more on performance than on new features, quality improvements and security updates. Announced at the company's Worldwide Developers Conference on June 4, 2018, iOS 12 was released to the public on September 17, 2018. It was succeeded for the iPhone and iPod Touch by iOS 13 on September 19, 2019, and for the iPad by iPadOS 13 on September 24, 2019. Security updates for iOS 12 continued for four years after the releases of iOS 13 and iPadOS 13 for devices unable to run the newer versions. The last update, 12.5.7, was released on January 23, 2023.

iOS 13 2019 mobile operating system

iOS 13 is the thirteenth major release of the iOS mobile operating system developed by Apple Inc. for the iPhone, iPod Touch and HomePod. The successor to iOS 12, it was announced at the company's Worldwide Developers Conference (WWDC) on June 3, 2019, and released on September 19, 2019. It was succeeded by iOS 14, released on September 16, 2020.

iOS 14 2020 mobile operating system

iOS 14 is the fourteenth major release of the iOS mobile operating system developed by Apple Inc. for the iPhone and iPod touch lines. Announced at the company's Worldwide Developers Conference on June 22, 2020 as the successor to iOS 13, it was released to the public on September 16, 2020. It was succeeded by iOS 15 on September 20, 2021.

iPhone 12 14th-generation smartphone produced by Apple Inc.

The iPhone 12 and iPhone 12 Mini are smartphones designed, developed, and marketed by Apple Inc. They are the fourteenth-generation iPhones, succeeding the iPhone 11. They were unveiled at a virtually held Apple Special Event at Apple Park in Cupertino, California, on October 13, 2020, alongside the "premium flagship" iPhone 12 Pro and iPhone 12 Pro Max and HomePod Mini. Pre-orders for the iPhone 12 started on October 16, 2020, and the phone was released in most countries on October 23, 2020, alongside the iPhone 12 Pro and fourth-generation iPad Air. Pre-orders for the iPhone 12 Mini began on November 6, 2020, and the phone was released on November 13, 2020, alongside the iPhone 12 Pro Max.

The iOS operating system utilizes many security features in both hardware and software, from the boot process to biometrics.

References

  1. "Apple's Face ID: Cheat sheet". TechRepublic. June 11, 2020. Retrieved 2020-12-07.
  2. "About Face ID advanced technology". Apple Support. Retrieved 2020-12-07.
  3. "Apple awards Finisar $390 million from its Advanced Manufacturing Fund". apple.com. vertical-cavity surface-emitting lasers (VCSELs) power some of Apple's most popular new features, including Face ID
  4. "Apple Special Event 2017". YouTube .
  5. 1 2 Collins, Katie (October 20, 2020). "Our masks make Face ID useless. iPhone 12 did nothing to fix it". CNET. Retrieved October 20, 2020.
  6. Asmelash, Leah (11 August 2020). "New York's MTA is asking Apple to create a Face ID that works with masks". CNN. Retrieved 10 November 2020.
  7. 1 2 "iPhone update lets Apple Watch users unlock Face ID in a mask". The Guardian . 2021-02-02. Retrieved 2021-02-03.
  8. 1 2 "Use Face ID while wearing a mask with iPhone 12 and later". Apple Support. 2022-03-14. Retrieved 2022-03-26.
  9. Savov, Vlad (September 12, 2017). "iPhone X announced with edge-to-edge screen, Face ID, and no home button". The Verge . Vox Media . Retrieved December 13, 2017.
  10. 1 2 Tepper, Fitz (September 12, 2017). "Face ID is replacing Touch ID on the new iPhone X". TechCrunch . Oath Inc. Retrieved December 13, 2017.
  11. "Apple's new iPad Pro has Face ID, USB-C, and slimmer bezels than ever before". The Verge. Retrieved 2018-11-20.
  12. Koetsier, John (September 23, 2019). "Apple Finally Gets Face Unlock Right: iPhone 11 And iOS 13". Forbes. Retrieved October 31, 2020.
  13. Buckley, Sean (November 17, 2013). "Report: Apple buys PrimeSense, co-creators of the original Kinect". Engadget . Retrieved November 4, 2020.
  14. Cooper, Daniel (November 4, 2020). "Ten years on, Kinect's legacy goes beyond Xbox". Engadget . Retrieved November 4, 2020.
  15. Kubota, Yoko (September 27, 2017). "Apple iPhone X Production Woe Sparked by Juliet and Her Romeo". The Wall Street Journal . Archived from the original on September 28, 2017. Retrieved September 27, 2017.
  16. "Apple iPhone X - IR Dot Projector" (PDF). To provide the 30,000 dots, the VCSEL supplies the IR light and the Folded Optic directs the IR light to the Active Diffractive Optical Element (DOE). Finally, the Active DOE divides the light beam into 30,000 dots of light.
  17. 1 2 3 Brandom, Russell (September 12, 2017). "The five biggest questions about Apple's new facial recognition system". The Verge . Vox Media. Archived from the original on November 15, 2017. Retrieved November 9, 2017.
  18. Ng, Alfred (September 27, 2017). "Is Face ID secure? Apple takes on lingering questions". CNET . CBS Interactive. Archived from the original on October 2, 2017. Retrieved November 9, 2017.
  19. Panzarino, Matthew (September 15, 2017). "Interview: Apple's Craig Federighi answers some burning questions about Face ID". TechCrunch . Oath Inc. Retrieved December 13, 2017.
  20. Warren, Tom (October 25, 2017). "Apple's Face ID struggles detailed in new iPhone X report". The Verge . Vox Media . Retrieved December 13, 2017.
  21. Clover, Juli (September 13, 2017). "Apple's New Face ID Biometric System Works in the Dark and When Your Face is Obscured by Hats and Beard". MacRumors . Retrieved December 13, 2017.
  22. Clover, Juli (September 14, 2017). "Apple's Face ID Feature Works With Most Sunglasses, Can Be Quickly Disabled to Thwart Thieves". MacRumors . Retrieved December 13, 2017.
  23. Heisler, Yoni (November 3, 2017). "Infrared video shows off the iPhone X's new Face ID feature in action". BGR . Penske Media Corporation . Retrieved December 19, 2017.
  24. "Eye-tracking app Eyeware Beam free to download in iPhone beta". Apple Insider. 29 June 2021. Retrieved 2021-09-08.
  25. "About Face ID advanced technology". Apple Support. November 7, 2018. Retrieved March 7, 2019.
  26. Ocbazghi, Emmanuel (October 31, 2017). "We put the iPhone X's Face ID to the ultimate test with identical twins — and the results surprised us". Business Insider . Axel Springer SE . Retrieved December 4, 2017.
  27. Ulanoff, Lance (October 31, 2017). "The iPhone X can't tell the difference between identical twins". Mashable . Retrieved December 4, 2017.
  28. Deahl, Dani (November 14, 2017). "This 10-year-old was able to unlock his mom's iPhone using Face ID". The Verge . Vox Media . Retrieved December 13, 2017.
  29. Hall, Zac (September 27, 2017). "Apple explains how iPhone X facial recognition with Face ID works (and fails) in security paper". 9to5Mac. Retrieved December 13, 2017.
  30. Kircher, Madison Malone (September 12, 2017). "Yes, You Can Unlock the New iPhone With Your Face. Here's How It Works". Select All. New York. Archived from the original on October 13, 2017. Retrieved November 9, 2017.
  31. Romm, Tony (September 13, 2017). "Apple is facing questions from the U.S. Senate on the privacy protections in iPhone X and Face ID". Recode . Vox Media. Archived from the original on November 10, 2017. Retrieved November 9, 2017.
  32. Lomas, Natasha (October 17, 2017). "Apple responds to Senator Franken's Face ID privacy concerns". TechCrunch . Oath Inc. Archived from the original on November 9, 2017. Retrieved November 9, 2017.
  33. Shah, Saqib (October 18, 2017). "Apple responds to Sen. Al Franken's Face ID concerns in letter". Engadget . Oath Inc. Archived from the original on November 10, 2017. Retrieved November 9, 2017.
  34. Robertson, Adi (September 12, 2017). "Why Face ID won't give you the legal protection of a passcode". The Verge . Vox Media . Retrieved December 4, 2017.
  35. Brewster, Thomas. "Feds Force Suspect To Unlock An Apple iPhone X With Their Face". Forbes. Retrieved 2019-05-05.
  36. Greenberg, Andy (November 3, 2017). "We tried really hard to beat Face ID - and failed (so far)". Wired . Condé Nast . Retrieved December 4, 2017.
  37. Greenberg, Andy (November 12, 2017). "Hackers say they've broken Face ID a week after iPhone X release". Wired . Condé Nast . Retrieved December 4, 2017.
  38. Ong, Thuy (November 13, 2017). "This $150 mask beat Face ID on the iPhone X". The Verge . Vox Media . Retrieved December 4, 2017.
  39. Nellis, Stephen (November 2, 2017). "App developer access to iPhone X face data spooks some privacy experts". Reuters . Thomson Reuters . Retrieved December 4, 2017.
  40. Statt, Nick (November 2, 2017). "Apple will share face mapping data from the iPhone X with third-party app developers". The Verge . Vox Media . Retrieved December 4, 2017.
  41. Bloom, Jonathan (May 21, 2020). "Face ID With a Mask? Here's How to Unlock Your iPhone in the Age of COVID-19". NBC Bay Area. Retrieved October 20, 2020.
  42. Coggan, Georgia (May 4, 2020). "Apple finally reveals Face ID 'fix' so you can keep your face mask on". Creative Bloq. Retrieved October 20, 2020.
  43. "Get to Know Face ID on iPhone". shop.rewa.tech. Retrieved 2022-08-21.
  44. Tibken, Shara (October 13, 2020). "iPhone 12 and the Touch ID button: Why Apple probably won't give it to us this year". CNET. Retrieved October 21, 2020.