Face Recognition Vendor Test

Last updated
FRVTlogo2006square.png

The Face Recognition Vendor Test (FRVT) was a series of large scale independent evaluations for face recognition systems realized by the National Institute of Standards and Technology in 2000, 2002, 2006, 2010, 2013 and 2017. Previous evaluations in the series were the Face Recognition Technology (FERET) evaluations in 1994, 1995 and 1996. The project is now in an Ongoing status with periodic reports, [1] and continues to grow in scope. It now includes tests for Face-in-Video-Evaluation (FIVE), facial morphing detection, and testing for demographic effects (e.g., age, gender, and race). [2]

Contents

FRVT 2006

The primary goal of the FRVT 2006 was to measure progress of prototype systems/algorithms and commercial face recognition systems since FRVT 2002. FRVT 2006 evaluated performance on:

To guarantee an accurate assessment, the FRVT 2006 measured performance with sequestered data (data not previously seen by the researchers or developers). A standard dataset and test methodology was employed so that all participants were evenly evaluated. The government provided both the test data and the test environment to participants. The test environment was called the Biometric Experimentation Environment (BEE). The BEE was the FRVT 2006 infrastructure. It allowed the experimenter to focus on the experiment by simplifying test data management, experiment configuration, and the processing of results.

The FRVT 2006 was sponsored by multiple U.S. Government agencies and was conducted and managed by the National Institute of Standards and Technology (NIST).

One of the goals of the FRVT 2006 was to independently determine if the objectives of the Face Recognition Grand Challenge (FRGC) were achieved. The FRGC was a separate algorithm development project designed to promote and advance face recognition technology that supports existing face recognition efforts in the U.S. Government. One of the objectives of the FRGC was to develop face recognition algorithms capable of performance an order of magnitude better than FRVT 2002. The FRGC was conducted from May 2004 through March 2006. FRGC data is still available to face recognition researchers. To obtain FRGC data, potential participants must sign the required licenses and follow FRGC data release rules. To request a FRGC data set, follow directions found on the "FRGC Webpage".

FRVT 2006 Protocol

FRVT 2006 Protocol

FRVT 2006 Executable Calling Signatures

FRVT 2006 Executable Naming Convention

FRVT 2006 results

The FRVT 2006 large-scale results are available in the combined FRVT 2006 and ICE 2006 Large-Scale Results evaluation report. It assesses algorithms based on input from 22 organizations in 10 different countries, with many submitting multiple algorithms. However, only those who successfully completed the large-scale tests are documented in this report. The report shows how error rates for the best algorithms have improved by orders of magnitude over the years, from a False Rejection Rate (FRR) of 0.79 at a False Acceptance Rate (FAR) of 0.001 in 1993, to a FRR=0.01 for FAR=0.001 in 2006. Part of this improvement is due to higher quality face images. The best results from 2006 were for "very high-resolution still images" (6 Mp) and 3D images. [3]

Face Recognition Prize Challenge 2017

The Face Recognition Prize Challenge (FRPC) assessed face recognition algorithms on photographs collected without tight quality constraints, e.g. images collected from individuals who are not cooperating or do not know they are photographed. Prized were awarded both for verification and identification. The best verification algorithm had a false non-match rate FNMR of 0.22 at a false match rate FMR of 0.001. Prizes were also awarded for speed and for verification against a set of cooperative portrait photos. [4]

FRVT Ongoing

FRVT Ongoing now has roughly 200 face recognition algorithms and tests against at least six collections of photographs [5] with multiple photographs of more than 8 million people. The best algorithms for 1:1 verification gives False Non Match Rates of 0.0003 at False Match Rates of 0.0001 on high quality visa images. [6]

Additional programs:

Sponsors

Related Research Articles

Biometrics are body measurements and calculations related to human characteristics. Biometric authentication is used in computer science as a form of identification and access control. It is also used to identify individuals in groups that are under surveillance.

<span class="mw-page-title-main">Iris recognition</span> Method of biometric identification

Iris recognition is an automated method of biometric identification that uses mathematical pattern-recognition techniques on video images of one or both of the irises of an individual's eyes, whose complex patterns are unique, stable, and can be seen from some distance. The discriminating powers of all biometric technologies depend on the amount of entropy they are able to encode and use in matching. Iris recognition is exceptional in this regard, enabling the avoidance of "collisions" even in cross-comparisons across massive populations. Its major limitation is that image acquisition from distances greater than a meter or two, or without cooperation, can be very difficult. However, the technology is in development and iris recognition can be accomplished from even up to 10 meters away or in a live camera feed.

<span class="mw-page-title-main">Facial recognition system</span> Technology capable of matching a face from an image against a database of faces

A facial recognition system is a technology potentially capable of matching a human face from a digital image or a video frame against a database of faces. Such a system is typically employed to authenticate users through ID verification services, and works by pinpointing and measuring facial features from a given image.

<span class="mw-page-title-main">Face detection</span> Identification of human faces in images

Face detection is a computer technology being used in a variety of applications that identifies human faces in digital images. Face detection also refers to the psychological process by which humans locate and attend to faces in a visual scene.

<span class="mw-page-title-main">Three-dimensional face recognition</span> Mode of facial recognition

Three-dimensional face recognition is a modality of facial recognition methods in which the three-dimensional geometry of the human face is used. It has been shown that 3D face recognition methods can achieve significantly higher accuracy than their 2D counterparts, rivaling fingerprint recognition.

The Facial Recognition Technology (FERET) database is a dataset used for facial recognition system evaluation as part of the Face Recognition Technology (FERET) program. It was first established in 1993 under a collaborative effort between Harry Wechsler at George Mason University and Jonathan Phillips at the Army Research Laboratory in Adelphi, Maryland. The FERET database serves as a standard database of facial images for researchers to use to develop various algorithms and report results. The use of a common database also allowed one to compare the effectiveness of different approaches in methodology and gauge their strengths and weaknesses.

Facial motion capture is the process of electronically converting the movements of a person's face into a digital database using cameras or laser scanners. This database may then be used to produce computer graphics (CG), computer animation for movies, games, or real-time avatars. Because the motion of CG characters is derived from the movements of real people, it results in a more realistic and nuanced computer character animation than if the animation were created manually.

The histogram of oriented gradients (HOG) is a feature descriptor used in computer vision and image processing for the purpose of object detection. The technique counts occurrences of gradient orientation in localized portions of an image. This method is similar to that of edge orientation histograms, scale-invariant feature transform descriptors, and shape contexts, but differs in that it is computed on a dense grid of uniformly spaced cells and uses overlapping local contrast normalization for improved accuracy.

Private biometrics is a form of encrypted biometrics, also called privacy-preserving biometric authentication methods, in which the biometric payload is a one-way, homomorphically encrypted feature vector that is 0.05% the size of the original biometric template and can be searched with full accuracy, speed and privacy. The feature vector's homomorphic encryption allows search and match to be conducted in polynomial time on an encrypted dataset and the search result is returned as an encrypted match. One or more computing devices may use an encrypted feature vector to verify an individual person or identify an individual in a datastore without storing, sending or receiving plaintext biometric data within or between computing devices or any other entity. The purpose of private biometrics is to allow a person to be identified or authenticated while guaranteeing individual privacy and fundamental human rights by only operating on biometric data in the encrypted space. Some private biometrics including fingerprint authentication methods, face authentication methods, and identity-matching algorithms according to bodily features. Private biometrics are constantly evolving based on the changing nature of privacy needs, identity theft, and biotechnology.

VeriFace is an online service for remote identity verification of customers developed by Slovak company Rokan Biometrics. It aims to provide biometric identity verification and customer onboarding via facial recognition software.

The Facial Recognition Technology (FERET) program was a government-sponsored project that aimed to create a large, automatic face-recognition system for intelligence, security, and law enforcement purposes. The program began in 1993 under the combined leadership of Dr. Harry Wechsler at George Mason University (GMU) and Dr. Jonathan Phillips at the Army Research Laboratory (ARL) in Adelphi, Maryland and resulted in the development of the Facial Recognition Technology (FERET) database. The goal of the FERET program was to advance the field of face recognition technology by establishing a common database of facial imagery for researchers to use and setting a performance baseline for face-recognition algorithms.

<span class="mw-page-title-main">Multiple Biometric Grand Challenge</span> Biometric project

Multiple Biometric Grand Challenge (MBGC) is a biometric project. Its primary goal is to improve performance of face and iris recognition technology on both still and video imagery with a series of challenge problems and evaluation.

<span class="mw-page-title-main">Face Recognition Grand Challenge</span>

The Face Recognition Grand Challenge (FRGC) was conducted from May 2004 until March 2006 to promote and advance face recognition technology. It succeeded the Face Recognition Vendor Test.

The Iris Challenge Evaluation (ICE) were a series of events conducted and managed by the National Institute of Standards and Technology for projects on technology development and evaluation for iris recognition. Industry, academia, and research institutions from both in and outside the U.S. were allowed to participate.

Identity-based security is a type of security that focuses on access to digital information or services based on the authenticated identity of an entity. It ensures that the users and services of these digital resources are entitled to what they receive. The most common form of identity-based security involves the login of an account with a username and password. However, recent technology has evolved into fingerprinting or facial recognition.

DeepFace is a deep learning facial recognition system created by a research group at Facebook. It identifies human faces in digital images. The program employs a nine-layer neural network with over 120 million connection weights and was trained on four million images uploaded by Facebook users. The Facebook Research team has stated that the DeepFace method reaches an accuracy of 97.35% ± 0.25% on Labeled Faces in the Wild (LFW) data set where human beings have 97.53%. This means that DeepFace is sometimes more successful than human beings. As a result of growing societal concerns Meta announced that it plans to shut down Facebook facial recognition system, deleting the face scan data of more than one billion users. This change will represent one of the largest shifts in facial recognition usage in the technology's history. Facebook planned to delete by December 2021 more than one billion facial recognition templates, which are digital scans of facial features. However, it did not plan to eliminate DeepFace which is the software that powers the facial recognition system. The company has also not ruled out incorporating facial recognition technology into future products, according to Meta spokesperson.

<span class="mw-page-title-main">Visage SDK</span> Software development kit

Visage SDK is a multi-platform software development kit (SDK) created by Visage Technologies AB. Visage SDK allows software programmers to build facial motion capture and eye tracking applications.

FindFace is a face recognition technology developed by the Russian company NtechLab that specializes in neural network tools. The company provides a line of services for the state and various business sectors based on FindFace algorithm. Previously, the technology was used as a web service that helped to find people on the VK social network using their photos.

Amazon Rekognition is a cloud-based software as a service (SaaS) computer vision platform that was launched in 2016. It has been sold to, and used by, a number of United States government agencies, including U.S. Immigration and Customs Enforcement (ICE) and Orlando, Florida police, as well as private entities.

DataWorks Plus LLC is a privately held biometrics systems integrator based in Greenville, South Carolina. The company started in 2000 and originally focused on mugshot management, adding facial recognition beginning in 2005. Brad Bylenga is the CEO, and Todd Pastorini is the EVP and GM. Usage of the technology by police departments has resulted in wrongful arrests.

References

  1. ↑ meilee.ngan@nist.gov (2016-12-14). "Face Recognition Vendor Test (FRVT) Ongoing". NIST. Retrieved 2020-02-22.
  2. ↑ patricia.flanagan@nist.gov (2010-07-08). "Face Recognition Vendor Test (FRVT)". NIST. Retrieved 2020-02-22.
  3. ↑ FRVT 2006 and ICE 2006 Large-Scale Results, March 2007, Figure 1 on p. 5, chapter 3 regarding image quality
  4. ↑ "Nist Interagency Report 8197" (PDF). NIST. 2017-11-22.
  5. ↑ Grother, Patrick; Ngan, Mei; Hanaoka, Kayee (19 November 2019). "Ongoing Face Recognition Vendor Test (FRVT) Part 1: Verification" (PDF). National Institute of Standards and Technology. Retrieved 22 February 2020.
  6. ↑ FRVT 1:1 Verification, as of 25 March 2000]
  7. ↑ Grother, Patrick; Ngan, Mei; Hanaok, Kayee (December 2019). "Face Recognition Vendor Test (FRVT) Chapter 3: Demographic Effects" (PDF). doi:10.6028/NIST.IR.8280.
  8. ↑ "FATE MORPH". NIST.
  9. ↑ "Face Challenges". NIST.
  10. ↑ "Face in Video Evaluation (FIVE)". NIST– via www.nist.gov.
This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.