Designing Custom Face Detection & Recognition Software

Recognition Software Over the last few months, one of our teams was busy designing a solution that employs biometric verification and works within real-time video camera streaming. The system workflow had to comprise 3 main stages: detection of a face, near-instant recognition, and further action to be taken (e.g. grant access to authorized personnel or send a corresponding alert otherwise).

Such is the research project that we called Big Brother. Its all-seeing eye uses computer vision to analyze frames of a stream and detect faces. See how it works in the video below.

Recognition Software

The potential of this solution can be fully realized in your own custom product, so feel free to address us with your ideas. Meanwhile we’ll reveal a few technical details on Big Brother’s workflow and constituents.

Face detection and recognition workflow

The entire process is triggered by a camera application, which is installed on a device that’s connected to a camera. This application was written in Golang as a local console app for Ubuntu and Raspbian. Upon launch, it must be configured using a JSON config file with Local Camera ID and Camera Reader type.

The camera app detects faces in the stream using a deep neural network and computer vision. During the research, we found that the two optimal ways to implement detection were Caffe face tracking and TensorFlow object detection models. Both of these were included in the OpenCV library and worked well when we tested them.

When the face is captured, the image is cropped and sent to the back end via HTTP form data request. The back end API saves the image to a local file system and saves a record to Detection Log with a personID.

Recognition Software1

The back end has a background worker that finds records with “classified = false” and uses Dlib to calculate the 128-dimensional descriptor vector of face features. Whenever a vector is calculated, it is compared with the existing entries (multiple reference face images) by calculating Euclidean distance to each feature vector of each Person in the database, finding a match, if any.
This is what it looks like in the code: each point index is properly defined in Dlib and represents various parts of a face.

Read More:- Really Useful Android Tips & Tricks no One Knows

If the Euclidean distance to the detected person is less than 0.6, the worker sets a personID to the Detection Log and marks it as classified. If the distance exceeds 0.6, it creates a new Person with unknown type and enters a new personID to the log.

Images of an unidentified person can be sent to the corresponding manager with notifications—in this case, we implemented chatbots in messengers. Then it will be possible to grant remote access manually or take other actions.

Anti-spoofing measures to boost security of face recognition

Face recognition can be enhanced with auxiliary security measures. After all, a facial image is the easiest biometric identifier one can get, way easier than getting a fingerprint or retina scanning. This is where it becomes vital to take additional anti-spoofing features so that no one could get access by showing a picture of a face.

Here we can apply special algorithms based on a combination of data science, computer vision and deep learning:

1. Detection of eye movement: for example, a person blinks 15-30 times per minute. Thus we can check face liveness.

2. Texture analysis: we can get hand-crafted features and detect the difference between a real face and a fake one, which typically has less elaborate details, noise and other image artifacts. With these features at hand, we can apply support-vector machines for binary classification.

3. Active camera flash: it easily helps to tell the difference between a real face with complex geometry and an image with plain surface.

4. Challenge-response protocol: give instructions to the user—e.g. wink, smile, say something, or turn their head—and, again, see whether the face is real.

5. These ways can be combined. For example, we can use both challenge-response and texture analysis to counteract both photo- and video-spoofing.

The Big Brother project allowed us to work out the algorithms and mechanisms that will enable faster development of custom solutions for face-based authentication. That said, feel free to address us with any questions regarding Machine Learning consulting services.