FaceNet

Last updated

FaceNet is a facial recognition system developed by Florian Schroff, Dmitry Kalenichenko and James Philbin, a group of researchers affiliated with Google. The system was first presented at the 2015 IEEE Conference on Computer Vision and Pattern Recognition. [1] The system uses a deep convolutional neural network to learn a mapping (also called an embedding) from a set of face images to a 128-dimensional Euclidean space, and assesses the similarity between faces based on the square of the Euclidean distance between the images' corresponding normalized vectors in the 128-dimensional Euclidean space. The system uses the triplet loss function as its cost function and introduced a new online triplet mining method. The system achieved an accuracy of 99.63%, which is the highest score to date on the Labeled Faces in the Wild dataset using the unrestricted with labeled outside data protocol. [2]

Contents

Structure

Basic structure

The structure of FaceNet is represented schematically in Figure 1.

Figure 1: Overall structure of the FaceNet face recognition system Structure of FaceNet System.png
Figure 1: Overall structure of the FaceNet face recognition system

For training, researchers used input batches of about 1800 images. For each identity represented in the input batches, there were 40 similar images of that identity and several randomly selected images of other identities. These batches were fed to a deep convolutional neural network, which was trained using stochastic gradient descent with standard backpropagation and the Adaptive Gradient Optimizer (AdaGrad) algorithm. The learning rate was initially set at 0.05, which was later lowered while finalizing the model.

Structure of the CNN

The researchers used two types of architectures, which they called NN1 and NN2, and explored their trade-offs. The practical differences between the models lie in the difference of parameters and FLOPS. The details of the NN1 model are presented in the table below.

Structure of the CNN used in the model NN1 in the FaceNet face recognition system
LayerSize-in
(rows × cols × #filters)
Size-out
(rows × cols × #filters)
Kernel
(rows × cols, stride)
ParametersFLOPS
conv1220×220×3110×110×647×7×3, 29K115M
pool1110×110×6455×55×643×3×64, 20
rnorm155×55×6455×55×640
conv2a55×55×6455×55×641×1×64, 14K13M
conv255×55×6455×55×1923×3×64, 1111K335M
rnorm255×55×19255×55×1920
pool255×55×19228×28×1923×3×192, 20
conv3a28×28×19228×28×1921×1×192, 137K29M
conv328×28×19228×28×3843×3×192, 1664K521M
pool328×28×38414×14×3843×3×384, 20
conv4a14×14×38414×14×3841×1×384, 1148K29M
conv414×14×38414×14×2563×3×384, 1885K173M
conv5a14×14×25614×14×2561×1×256, 166K13M
conv514×14×25614×14×2563×3×256, 1590K116M
conv6a14×14×25614×14×2561×1×256, 166K13M
conv614×14×25614×14×2563×3×256, 1590K116M
pool414×14×2563×3×256, 27×7×2560
concat7×7×2567×7×2560
fc17×7×2561×32×128maxout p=2103M103M
fc21×32×1281×32×128maxout p=234M34M
fc71281×32×1281×1×128524K0.5M
L21×1×1281×1×1280
Total140M1.6B

Triplet loss

A key innovation of the system was the triplet loss function and its associated mining method. This function has since become central in a variety of other one-shot learning problems.

Performance

On the widely used Labeled Faces in the Wild (LFW) dataset, the FaceNet system achieved an accuracy of 99.63% which is the highest score on LFW in the unrestricted with labeled outside data protocol. [2] On YouTube Faces DB the system achieved an accuracy of 95.12%. [1]

See also

Further reading

References

  1. 1 2 Florian Schroff; Dmitry Kalenichenko; James Philbin. "FaceNet: A Unified Embedding for Face Recognition and Clustering" (PDF). The Computer Vision Foundation. Retrieved 4 October 2023.
  2. 1 2 Erik Learned-Miller; Gary Huang; Aruni RoyChowdhury; Haoxiang Li; Gang Hua (April 2016). "Labeled Faces in the Wild: A Survey". Advances in Face Detection and Facial Image Analysis (PDF). Springer. pp. 189–248. Retrieved 5 October 2023.