What’s the Best Face Detector?. Comparing Dlib, OpenCV, MTCNN, and… | by Amos Stailey-Young | Jun, 2024 – Niraranra – Niraranra

Evaluating Dlib, OpenCV DNN, Yunet, Pytorch-MTCNN, and RetinaFace

For a facial recognition draw back I’m engaged on, I wanted to search out out which facial detection mannequin to pick out. Face detection is the primary a part of the facial recognition pipeline, and it’s crucial that the detector precisely identifies faces contained in the picture. Rubbish in, rubbish out, in the long run.

Nevertheless, the myriad alternatives accessible left me feeling overwhelmed, and the scattered writings on the topic weren’t detailed ample to assist me resolve on a mannequin. Evaluating the quite a few fashions took an entire lot of labor, so I figured relaying my analysis may assist people in related conditions.

The first trade-off when choosing a facial detection mannequin is that between accuracy and effectivity. Nonetheless there are fully completely different components to ponder.

A great deal of the articles on face detection fashions are written every by the creators of the mannequin — usually in journals — or by these implementing the mannequin in code. In each circumstances, the writers, naturally, have a bias in route of the mannequin they’re writing about. In some excessive circumstances, they’re primarily promotional commercials for the mannequin in query.

There aren’t many articles that think about how the completely fully completely different fashions carry out in opposition to not less than one one different. Along with further confusion, every time any particular person is writing a few mannequin similar to RetinaFace, they’re speaking a few particular implementation of that mannequin. The “mannequin” itself is totally the neural group development, and completely fully completely different implementations of the an similar group development can result in completely fully completely different outcomes. To make factors additional powerful, the effectivity of those fashions furthermore differs based on post-processing parameters, similar to confidence thresholds, non-maximum suppression, and so forth.

Each author casts their mannequin on account of the “greatest”, nonetheless I shortly realized that “greatest” depends upon upon context. There isn’t any goal greatest mannequin. There are two primary necessities when deciding which face detection mannequin is most related for the given context: accuracy and tempo.

No mannequin combines excessive accuracy with excessive tempo; it’s a trade-off. We even have to try metrics earlier uncooked accuracy, on which most benchmarks are based (relevant guesses / full pattern measurement), nonetheless uncooked accuracy isn’t going to be the one metric to concentrate to. The ratio of false positives to true positives, and false negatives to true negatives, may very well be an crucial consideration. In technical phrases, the trade-off is between precision (minimizing false positives) and recall (minimizing false negatives). This article discusses the issue in further depth.

There are a couple of present face detection datasets used for benchmarking, similar to WIDER FACE, nonetheless I all the time wish to see how the fashions will carry out by myself information. So I randomly grabbed 1064 frames from my pattern of TV shows to check the fashions ( ±3% margin of error). When manually annotating every picture, I attempted to pick out as many faces as attainable, together with faces that had been partially or virtually fully occluded to provide the fashions an exact downside. Due to I’m lastly going to carry out facial recognition on the detected faces, I wished to check the boundaries of every mannequin.

The images may very well be found to amass with their annotations. I’ve furthermore shared a Google Colab pocket information to work together with the information here.

It helps to group the quite a few fashions into two camps; people who run on the GPU and individuals who run on the CPU. Typically, when you’ve purchased a CUDA-compatible GPU, it’s biggest to utilize a GPU-based mannequin. I’ve an NVIDIA 1080 TI with 11GB of reminiscence, which permits me to make the most of among the many many larger-scale fashions. Nonetheless, the scale of my enterprise is large (I’m speaking tons of of video recordsdata), so the lightning-fast CPU-based fashions intrigued me. There aren’t many CPU-based face detection fashions, so I made a decision to check solely primarily probably the most well-liked one: YuNet. On account of its tempo, YuNet kinds my baseline comparability. A GPU mannequin have to be considerably additional acceptable than its CPU counterpart to justify its slower processing tempo.

YuNet

YuNet was developed with effectivity in concepts with a mannequin measurement that’s solely a fraction of the bigger fashions. As an illustration, YuNet has solely 75,856 parameters as in contrast with the 27,293,600 that RetinaFace boasts, which permits YuNet to run on “edge” computing objects that aren’t extraordinarily environment friendly ample for the bigger fashions.

Code to implement the YuNet mannequin will be discovered on this repository. The one strategy to get YuNet up and dealing is thru OpenCV.

cv2.FaceDetectorYN_create('./face_detection_yunet_2023mar.onnx',
"", 
(300, 300),
score_threshold=0.5) 

The pre-trained mannequin is available on the market on the OpenCV Zoo repository here. Merely ensure when cloning the repo to make the most of Git LFS (I made that mistake at first). There’s a Google Colab file I wrote to indicate accessible here.

YuNet carried out so much bigger than I anticipated for a CPU mannequin. It’s in a position to detect large faces and by no means using a draw back nonetheless does wrestle a bit with smaller ones.

The accuracy improves enormously when limiting to the most important face contained in the picture.

If effectivity is a critical concern, YuNet is an efficient danger. It’s even quick ample for real-time features, in distinction to the GPU alternatives accessible (on the very least with out some necessary {{{hardware}}}).

Dlib

Dlib is a C++-implementation with a Python wrapper that maintains a stability between accuracy, effectivity, and luxurious. Dlib will be put in instantly by means of Python or accessed by means of the Face Recognition Python library. Nevertheless, there’s a very sturdy trade-off between Dlib accuracy and effectivity based completely on the upsampling parameter. When the variety of circumstances to upsample is about to 0, the mannequin is quicker nonetheless so much a lot much less acceptable.

No Upsampling

Upsampling = 1

The accuracy of the Dlib mannequin will improve with further upsampling, nonetheless one factor elevated than upsampling=1 would set off my script to crash due to it exceeded my GPU reminiscence (which is 11GB by the best way during which by way of which).

Dlib’s accuracy was considerably disappointing relative to its (lack of) tempo. Nevertheless, it was fantastic at minimizing false positives, which is a precedence of mine. Face detection is the primary a part of my facial recognition pipeline, so minimizing the variety of false positives will assist scale back errors downstream. To scale back the variety of false positives even further, we’ll use Dlib’s confidence output to filter lower-confident samples.

There’s a big discrepancy in confidence between false and true positives, which we’ll use to filter out the sooner. Pretty than select an arbitrary threshold, we’ll have a look on the distribution of confidence scores to pick out an extra precise one.

95% of the boldness values fall above 0.78, so excluding every half beneath that worth reduces the variety of false positives by half.

Whereas filtering by confidence reduces the variety of false positives, it doesn’t enhance the ultimate accuracy. I’d think about using Dlib when minimizing the variety of false positives is a critical concern. Nonetheless in each different case, Dlib doesn’t present an unlimited ample enhance in accuracy over YuNet to justify the a lot elevated processing circumstances; on the very least for my capabilities.

OpenCV DNN

The first draw of OpenCV’s face detection mannequin is its tempo. Nevertheless, its accuracy left one issue to be desired. Whereas it’s terribly quick when as in contrast with the opposite GPU fashions, even its Prime 1 accuracy was hardly bigger than YuNet’s full accuracy. It’s unclear to me all through which state of affairs I’d ever select the OpenCV mannequin for face detection, considerably because it may very well be powerful to get working (you will need to assemble OpenCV from present, which I’ve written about here).

Pytorch-MCNN

The MTCNN mannequin furthermore carried out fairly poorly. Though it was barely additional acceptable than the OpenCV mannequin, it was fairly a bit slower. Since its accuracy was decrease than YuNet, there was no compelling set off to pick out MTCNN.

RetinaFace

RetinaFace has a status for being principally primarily probably the most acceptable of open-source face detection fashions. The take a look at outcomes as soon as extra up that status.

Not solely was it principally primarily probably the most acceptable mannequin, nonetheless lots of the “inaccuracies” weren’t, in fact, precise errors. RetinaFace actually examined the class of “false constructive” on account of it picked up faces I hadn’t seen, hadn’t bothered to annotate due to I believed them too troublesome, or hadn’t even thought-about a “face.”

It picked up a partial face in a mirror on this Seinfeld physique.

It managed to look out faces in image frames contained in the background of this Trendy Household.

And it’s so good at figuring out “faces,” that it finds non-human ones.

It was a lovely shock discovering out that RetinaFace wasn’t all that gradual every. Whereas it wasn’t as quick as YuNet or OpenCV, it was similar to MTCNN. Whereas it’s slower at decrease resolutions than MTCNN, it scales comparatively correctly and will course of elevated resolutions merely as shortly. And RetinaFace beat Dlib (on the very least when having to upscale). It’s a lot slower than YuNet nonetheless is considerably additional acceptable.

Loads of the “false positives” RetinaFace acknowledged will be excluded by filtering out smaller faces. If we drop the underside quartile of faces, the false constructive cost drops drastically.

Whereas RetinaFace is awfully acceptable, the errors do have a selected bias. Though RetinaFace identifies small faces with ease, it struggles with bigger, partially occluded ones, which is clear if we check out face measurement relative to accuracy.

This may in all probability be problematic for my capabilities on account of the measurement of a face in a picture is strongly correlated to its significance. Subsequently, RetinaFace may miss an important circumstances, similar to the event beneath.

Based mostly completely on my exams (which I’d like to emphasise are sometimes not principally primarily probably the most rigorous on this planet; so take them with a grain of salt), I’d solely think about using every YuNet or RetinaFace, relying on whether or not or not or not tempo or accuracy was my foremost concern. It’s attainable I’d consider using Dlib if I completely wished to attenuate false positives, nonetheless for my enterprise, it’s correct proper right down to YuNet or RetinaFace.

The GitHub repo used for this enterprise is available on the market here.