What’s the Best Face Detector?. Comparing Dlib, OpenCV, MTCNN, and… | by Amos Stailey-Young | Jun, 2024 – Niraranra

Evaluating Dlib, OpenCV DNN, Yunet, Pytorch-MTCNN, and RetinaFace

For a facial recognition downside I’m engaged on, I needed to find out which facial detection model to select. Face detection is the first part of the facial recognition pipeline, and it’s very important that the detector exactly identifies faces inside the image. Garbage in, garbage out, in the end.

However, the myriad selections accessible left me feeling overwhelmed, and the scattered writings on the subject weren’t detailed ample to help me resolve on a model. Evaluating the numerous fashions took a complete lot of labor, so I figured relaying my evaluation might help individuals in associated situations.

The primary trade-off when selecting a facial detection model is that between accuracy and effectivity. Nevertheless there are completely different parts to ponder.

Loads of the articles on face detection fashions are written each by the creators of the model — generally in journals — or by these implementing the model in code. In every circumstances, the writers, naturally, have a bias in direction of the model they’re writing about. In some extreme circumstances, they’re mainly promotional advertisements for the model in question.

There aren’t many articles that consider how the utterly completely different fashions perform in opposition to at least one one other. Together with extra confusion, each time any individual is writing a couple of model just like RetinaFace, they’re talking a couple of specific implementation of that model. The “model” itself is completely the neural group construction, and utterly completely different implementations of the an identical group construction can lead to utterly completely different outcomes. To make points further tough, the effectivity of these fashions moreover differs primarily based on post-processing parameters, just like confidence thresholds, non-maximum suppression, and so forth.

Every writer casts their model as a result of the “best”, nonetheless I shortly realized that “best” relies upon upon context. There isn’t any objective best model. There are two basic requirements when deciding which face detection model is most relevant for the given context: accuracy and tempo.

No model combines extreme accuracy with extreme tempo; it’s a trade-off. We even have to take a look at metrics previous raw accuracy, on which most benchmarks are primarily based (applicable guesses / full sample measurement), nonetheless raw accuracy is not going to be the one metric to pay attention to. The ratio of false positives to true positives, and false negatives to true negatives, could be an very important consideration. In technical phrases, the trade-off is between precision (minimizing false positives) and recall (minimizing false negatives). This article discusses the difficulty in extra depth.

There are a few current face detection datasets used for benchmarking, just like WIDER FACE, nonetheless I always want to see how the fashions will perform on my own data. So I randomly grabbed 1064 frames from my sample of TV displays to test the fashions ( ±3% margin of error). When manually annotating each image, I tried to select as many faces as attainable, along with faces that had been partially or practically completely occluded to supply the fashions an precise drawback. Because of I’m finally going to hold out facial recognition on the detected faces, I wanted to test the boundaries of each model.

The images could be discovered to acquire with their annotations. I’ve moreover shared a Google Colab pocket guide to work along with the knowledge here.

It helps to group the numerous fashions into two camps; those that run on the GPU and people who run on the CPU. Often, while you’ve bought a CUDA-compatible GPU, it’s greatest to make use of a GPU-based model. I’ve an NVIDIA 1080 TI with 11GB of memory, which allows me to utilize among the many larger-scale fashions. Nevertheless, the size of my enterprise is giant (I’m talking tons of of video recordsdata), so the lightning-fast CPU-based fashions intrigued me. There aren’t many CPU-based face detection fashions, so I decided to test solely essentially the most well-liked one: YuNet. As a result of its tempo, YuNet sorts my baseline comparability. A GPU model must be significantly further appropriate than its CPU counterpart to justify its slower processing tempo.

YuNet

YuNet was developed with effectivity in ideas with a model measurement that’s solely a fraction of the larger fashions. For instance, YuNet has solely 75,856 parameters as compared with the 27,293,600 that RetinaFace boasts, which allows YuNet to run on “edge” computing items that aren’t extremely efficient ample for the larger fashions.

Code to implement the YuNet model can be found on this repository. The only approach to get YuNet up and dealing is through OpenCV.

cv2.FaceDetectorYN_create('./face_detection_yunet_2023mar.onnx',
"", 
(300, 300),
score_threshold=0.5) 

The pre-trained model is on the market on the OpenCV Zoo repository here. Merely make sure when cloning the repo to utilize Git LFS (I made that mistake at first). There’s a Google Colab file I wrote to show accessible here.

YuNet carried out a lot larger than I anticipated for a CPU model. It’s able to detect big faces and never utilizing a downside nonetheless does wrestle a bit with smaller ones.

The accuracy improves enormously when limiting to the largest face inside the image.

If effectivity is a serious concern, YuNet is an effective risk. It’s even fast ample for real-time functions, in distinction to the GPU selections accessible (on the very least with out some important {{hardware}}).

Dlib

Dlib is a C++-implementation with a Python wrapper that maintains a stability between accuracy, effectivity, and luxury. Dlib can be put in immediately by way of Python or accessed by way of the Face Recognition Python library. However, there’s a really sturdy trade-off between Dlib accuracy and effectivity primarily based totally on the upsampling parameter. When the number of cases to upsample is about to 0, the model is faster nonetheless a lot much less appropriate.

No Upsampling

Upsampling = 1

The accuracy of the Dlib model will enhance with extra upsampling, nonetheless one thing elevated than upsampling=1 would set off my script to crash because of it exceeded my GPU memory (which is 11GB by the way in which through which).

Dlib’s accuracy was significantly disappointing relative to its (lack of) tempo. However, it was wonderful at minimizing false positives, which is a priority of mine. Face detection is the first part of my facial recognition pipeline, so minimizing the number of false positives will help reduce errors downstream. To reduce the number of false positives even extra, we’ll use Dlib’s confidence output to filter lower-confident samples.

There’s a large discrepancy in confidence between false and true positives, which we’ll use to filter out the earlier. Fairly than choose an arbitrary threshold, we’ll take a look on the distribution of confidence scores to select a further actual one.

95% of the boldness values fall above 0.78, so excluding each half below that price reduces the number of false positives by half.

Whereas filtering by confidence reduces the number of false positives, it doesn’t improve the final accuracy. I’d consider using Dlib when minimizing the number of false positives is a serious concern. Nevertheless in every other case, Dlib doesn’t provide an enormous ample improve in accuracy over YuNet to justify the so much elevated processing cases; on the very least for my capabilities.

OpenCV DNN

The primary draw of OpenCV’s face detection model is its tempo. However, its accuracy left one factor to be desired. Whereas it’s extraordinarily fast when as compared with the other GPU fashions, even its Prime 1 accuracy was hardly larger than YuNet’s complete accuracy. It’s unclear to me throughout which state of affairs I’d ever choose the OpenCV model for face detection, significantly since it could be tough to get working (you must assemble OpenCV from provide, which I’ve written about here).

Pytorch-MCNN

The MTCNN model moreover carried out pretty poorly. Although it was barely further appropriate than the OpenCV model, it was pretty a bit slower. Since its accuracy was lower than YuNet, there was no compelling trigger to select MTCNN.

RetinaFace

RetinaFace has a reputation for being basically essentially the most appropriate of open-source face detection fashions. The check out outcomes once more up that reputation.

Not solely was it basically essentially the most appropriate model, nonetheless many of the “inaccuracies” weren’t, in actuality, exact errors. RetinaFace truly examined the category of “false constructive” as a result of it picked up faces I hadn’t seen, hadn’t bothered to annotate because of I believed them too troublesome, or hadn’t even considered a “face.”

It picked up a partial face in a mirror on this Seinfeld physique.

It managed to search out faces in picture frames inside the background of this Modern Family.

And it’s so good at determining “faces,” that it finds non-human ones.

It was a pleasing shock finding out that RetinaFace wasn’t all that gradual each. Whereas it wasn’t as fast as YuNet or OpenCV, it was just like MTCNN. Whereas it’s slower at lower resolutions than MTCNN, it scales comparatively properly and should course of elevated resolutions merely as shortly. And RetinaFace beat Dlib (on the very least when having to upscale). It’s so much slower than YuNet nonetheless is significantly further appropriate.

A lot of the “false positives” RetinaFace acknowledged can be excluded by filtering out smaller faces. If we drop the underside quartile of faces, the false constructive payment drops drastically.

Whereas RetinaFace is extraordinarily appropriate, the errors do have a particular bias. Although RetinaFace identifies small faces with ease, it struggles with larger, partially occluded ones, which is apparent if we take a look at face measurement relative to accuracy.

This might probably be problematic for my capabilities as a result of the measurement of a face in an image is strongly correlated to its significance. Subsequently, RetinaFace might miss an vital circumstances, just like the occasion below.

Based totally on my exams (which I’d like to emphasize are often not basically essentially the most rigorous on this planet; so take them with a grain of salt), I’d solely consider using each YuNet or RetinaFace, counting on whether or not or not tempo or accuracy was my main concern. It’s attainable I’d think about using Dlib if I utterly wanted to attenuate false positives, nonetheless for my enterprise, it’s proper right down to YuNet or RetinaFace.

The GitHub repo used for this enterprise is on the market here.