What’s the Best Face Detector?. Comparing Dlib, OpenCV, MTCNN, and… | by Amos Stailey-Young | Jun, 2024 – Niraranra – Niraranra – Niraranra – Niraranra

Evaluating Dlib, OpenCV DNN, Yunet, Pytorch-MTCNN, and RetinaFace

For a facial recognition draw again I’m engaged on, I needed to look out out which facial detection model to pick. Face detection is the first part of the facial recognition pipeline, and it’s essential that the detector exactly identifies faces contained within the image. Garbage in, garbage out, in the long term.

However, the myriad alternate options accessible left me feeling overwhelmed, and the scattered writings on the subject weren’t detailed ample to help me resolve on a model. Evaluating the fairly a couple of fashions took a whole lot of labor, so I figured relaying my evaluation might help individuals in associated circumstances.

The primary trade-off when selecting a facial detection model is that between accuracy and effectivity. Nonetheless there are absolutely fully completely different elements to ponder.

Quite a lot of the articles on face detection fashions are written each by the creators of the model — normally in journals — or by these implementing the model in code. In every circumstances, the writers, naturally, have a bias in route of the model they’re writing about. In some extreme circumstances, they’re primarily promotional commercials for the model in question.

There aren’t many articles that take into consideration how the fully absolutely fully completely different fashions perform in opposition to not lower than one one completely different. Together with additional confusion, each time any explicit particular person is writing a couple of model just like RetinaFace, they’re talking a couple of explicit implementation of that model. The “model” itself is completely the neural group growth, and fully absolutely fully completely different implementations of the an related group growth may end up in fully absolutely fully completely different outcomes. To make components extra highly effective, the effectivity of these fashions moreover differs primarily based on post-processing parameters, just like confidence thresholds, non-maximum suppression, and so forth.

Every writer casts their model on account of the “best”, nonetheless I shortly realized that “best” relies upon upon upon context. There isn’t any purpose best model. There are two main requirements when deciding which face detection model is most associated for the given context: accuracy and tempo.

No model combines extreme accuracy with extreme tempo; it’s a trade-off. We even should attempt metrics earlier raw accuracy, on which most benchmarks are primarily based (related guesses / full sample measurement), nonetheless raw accuracy isn’t going to be the one metric to pay attention to. The ratio of false positives to true positives, and false negatives to true negatives, might very effectively be an essential consideration. In technical phrases, the trade-off is between precision (minimizing false positives) and recall (minimizing false negatives). This article discusses the problem in additional depth.

There are a few current face detection datasets used for benchmarking, just like WIDER FACE, nonetheless I on a regular basis want to see how the fashions will perform on my own data. So I randomly grabbed 1064 frames from my sample of TV reveals to test the fashions ( ±3% margin of error). When manually annotating each image, I tried to pick as many faces as attainable, along with faces that had been partially or just about absolutely occluded to offer the fashions a precise draw back. As a consequence of I’m lastly going to hold out facial recognition on the detected faces, I needed to test the boundaries of each model.

The images might very effectively be discovered to amass with their annotations. I’ve moreover shared a Google Colab pocket data to work along with the data here.

It helps to group the fairly a couple of fashions into two camps; individuals who run on the GPU and people who run on the CPU. Usually, if you’ve bought a CUDA-compatible GPU, it’s largest to make the most of a GPU-based model. I’ve an NVIDIA 1080 TI with 11GB of memory, which allows me to benefit from among the many many many larger-scale fashions. Nonetheless, the dimensions of my enterprise is giant (I’m talking tons of of video recordsdata), so the lightning-fast CPU-based fashions intrigued me. There aren’t many CPU-based face detection fashions, so I decided to test solely primarily most likely probably the most well-liked one: YuNet. On account of its tempo, YuNet varieties my baseline comparability. A GPU model should be significantly extra acceptable than its CPU counterpart to justify its slower processing tempo.

YuNet

YuNet was developed with effectivity in ideas with a model measurement that’s solely a fraction of the larger fashions. As an illustration, YuNet has solely 75,856 parameters as in distinction with the 27,293,600 that RetinaFace boasts, which allows YuNet to run on “edge” computing objects that aren’t terribly atmosphere pleasant ample for the larger fashions.

Code to implement the YuNet model might be found on this repository. The one technique to get YuNet up and dealing is through OpenCV.

cv2.FaceDetectorYN_create('./face_detection_yunet_2023mar.onnx',
"", 
(300, 300),
score_threshold=0.5) 

The pre-trained model is obtainable available on the market on the OpenCV Zoo repository here. Merely guarantee when cloning the repo to benefit from Git LFS (I made that mistake at first). There’s a Google Colab file I wrote to point accessible here.

YuNet carried out a lot larger than I anticipated for a CPU model. It’s able to detect giant faces and not at all utilizing a draw again nonetheless does wrestle a bit with smaller ones.

The accuracy improves enormously when limiting to a very powerful face contained within the image.

If effectivity is a important concern, YuNet is an environment friendly hazard. It’s even fast ample for real-time options, in distinction to the GPU alternate options accessible (on the very least with out some crucial {{{{hardware}}}}).

Dlib

Dlib is a C++-implementation with a Python wrapper that maintains a stability between accuracy, effectivity, and splendid. Dlib might be put in immediately by the use of Python or accessed by the use of the Face Recognition Python library. However, there’s a really sturdy trade-off between Dlib accuracy and effectivity primarily based fully on the upsampling parameter. When the number of circumstances to upsample is about to 0, the model is faster nonetheless a lot quite a bit a lot much less acceptable.

No Upsampling

Upsampling = 1

The accuracy of the Dlib model will enhance with additional upsampling, nonetheless one issue elevated than upsampling=1 would set off my script to crash on account of it exceeded my GPU memory (which is 11GB by one of the best ways throughout which by means of which).

Dlib’s accuracy was significantly disappointing relative to its (lack of) tempo. However, it was unbelievable at minimizing false positives, which is a priority of mine. Face detection is the first part of my facial recognition pipeline, so minimizing the number of false positives will help reduce errors downstream. To reduce the number of false positives even additional, we’ll use Dlib’s confidence output to filter lower-confident samples.

There’s an enormous discrepancy in confidence between false and true positives, which we’ll use to filter out the earlier. Fairly than choose an arbitrary threshold, we’ll take a look on the distribution of confidence scores to pick an additional exact one.

95% of the boldness values fall above 0.78, so excluding each half beneath that price reduces the number of false positives by half.

Whereas filtering by confidence reduces the number of false positives, it doesn’t improve the last word accuracy. I’d consider using Dlib when minimizing the number of false positives is a important concern. Nonetheless in every completely different case, Dlib doesn’t current an infinite ample improve in accuracy over YuNet to justify the quite a bit elevated processing circumstances; on the very least for my capabilities.

OpenCV DNN

The primary draw of OpenCV’s face detection model is its tempo. However, its accuracy left one situation to be desired. Whereas it’s terribly fast when as in distinction with the alternative GPU fashions, even its Prime 1 accuracy was hardly larger than YuNet’s full accuracy. It’s unclear to me all via which state of affairs I’d ever choose the OpenCV model for face detection, significantly as a result of it might very effectively be highly effective to get working (you’ll need to assemble OpenCV from current, which I’ve written about here).

Pytorch-MCNN

The MTCNN model moreover carried out pretty poorly. Although it was barely extra acceptable than the OpenCV model, it was pretty a bit slower. Since its accuracy was lower than YuNet, there was no compelling set off to pick MTCNN.

RetinaFace

RetinaFace has a standing for being principally primarily most likely probably the most acceptable of open-source face detection fashions. The check out outcomes as quickly as further up that standing.

Not solely was it principally primarily most likely probably the most acceptable model, nonetheless a number of the “inaccuracies” weren’t, the truth is, exact errors. RetinaFace really examined the category of “false constructive” on account of it picked up faces I hadn’t seen, hadn’t bothered to annotate on account of I believed them too troublesome, or hadn’t even thought-about a “face.”

It picked up a partial face in a mirror on this Seinfeld physique.

It managed to look out faces in picture frames contained within the background of this Fashionable Family.

And it’s so good at determining “faces,” that it finds non-human ones.

It was a stunning shock discovering out that RetinaFace wasn’t all that gradual each. Whereas it wasn’t as fast as YuNet or OpenCV, it was just like MTCNN. Whereas it’s slower at lower resolutions than MTCNN, it scales comparatively appropriately and can course of elevated resolutions merely as shortly. And RetinaFace beat Dlib (on the very least when having to upscale). It’s quite a bit slower than YuNet nonetheless is significantly extra acceptable.

A great deal of the “false positives” RetinaFace acknowledged might be excluded by filtering out smaller faces. If we drop the underside quartile of faces, the false constructive value drops drastically.

Whereas RetinaFace is very acceptable, the errors do have a specific bias. Although RetinaFace identifies small faces with ease, it struggles with larger, partially occluded ones, which is obvious if we try face measurement relative to accuracy.

This will possibly be problematic for my capabilities on account of the measurement of a face in an image is strongly correlated to its significance. Subsequently, RetinaFace might miss an necessary circumstances, just like the occasion beneath.

Primarily based fully on my exams (which I’d like to stress are generally not principally primarily most likely probably the most rigorous on this planet; so take them with a grain of salt), I’d solely consider using each YuNet or RetinaFace, counting on whether or not or not or not or not tempo or accuracy was my foremost concern. It’s attainable I’d think about using Dlib if I fully wished to attenuate false positives, nonetheless for my enterprise, it’s appropriate correct proper all the way down to YuNet or RetinaFace.

The GitHub repo used for this enterprise is obtainable available on the market here.