Liveness Detection

Liveness detection is a Machine Learning based solution to determine the likelihood of an organization interfacing with a physically present human being and an inanimate spoof artifact  (such as printing face on paper, displaying a recorded face using mobile devices, using human masks). This technology will recommend how to handle spoof activity to reduce the risk of fraud activity.
This technology is deployed on API and SDK V2, check the Passive liveness documentation API  and SDK documentation for the integration step.
Passive liveness Check Flow
In order to evaluate the spoof check, our system will perform three general processes, there are:
1.Face appearance check: In this part, we ensure the face frame size is above 300 x 300 px, the input image contains a face object (face detection), ensure the face position (face pose), and ensure the face object is not occluded by other objects (face occlusion)
2.Image quality assessment check: Image Quality Assessment: checking certain good quality images on defined variables, such as Brightness level; Spotlight, Exposed level, Sharpness level (Blur or not), and Contrast.
3.Spoof component check: Checking spoof component on the characteristic of artifact, deformation, and the spoof edge to check the liveness score.
In the figure below, we show the processes to detect passive liveness in our system.
Face Liveness Check
Passive Liveness Requirement Check
Image Input Specification
The submitted image input should fulfill the minimum requirements below:
Image Setting
Recommendation
Minimum camera pixel
Above 2 MP
Image file size
Minimum size 100 KB and the maximum are 2 MB
Image compression recommendation
Bicubic, with minimum JPEG quality 80%
Face Appearance Check: Face Detection and Pose Check
The submitted Face input should fulfill the minimum requirements below:
Image Setting
Text
Minimum face area size
300 x 300 pixels, or adjust on 600 x 600 pixels 
Ideal face posture 
Frontal face 
Face orientation position 
Vertical 
Total face in the frame
1 face
Before processing the image, we will ensure there is a face object on the frame using face detection. If the image input has multiple faces, the algorithm will choose the main bigger face and will check the face pose using head poses analysis to make sure that a face object is in frontal face position.
The algorithm will detect the main bigger face on the frame
The algorithm will detect the main bigger face on the frame
The head pose estimation is needed on this part to prevent over tilted pose that might be reduced prediction accuracy for face occlusion and liveness check. On the other side, the head pose check might help to reject some spoof from paper or replay video.
Face Appearance Check: Face Occlusion Check
face occlusion is an algorithm to check whether the face is occluded by an object or any wearables. The facial areas that is checked by the occlusion are:
Forehead 
Left and right eyebrow 
Left and right eyes 
Left and right cheeks 
Nose 
Mouth
Image Quality Assessment
The image quality assessment will check the quality of the face area. Attributes that are checked by the algorithm consists of:
Sharpness: an image quality attribute to evaluate the face area is blurred or not. The value below 0.1 indicates a very blur image, the value between 0.3 to 0.4 indicates doubt are, best condition of the sharpness is above 0.4.
Contrast: an image quality attribute to evaluate the face area has enough contrast condition. The value below 0.5 indicates low contrast,  the value between 0.5 to 0,6 indicates doubt condition, enough contrast should be above 0.6.
Brightness: an image quality attribute to evaluate the face area has enough lightning condition. The value below 0.3 indicates a very dark condition, the value between 0.3 to 0.4 indicates doubtful condition, and best brightness condition is above 0.4.
Face Quality Attribute
Reject
Doubt
Accept
Sharpness 
s < 0.1 
0.1 ≤ s < 0.2 
s ≥ 0.2 
Contrast
s < 0.5 
0.5 ≤ s < 0.6 
s ≥ 0.6 
 Brightness
s < 0.3
0.3 ≤ s < 0.4
s ≥ 0.4
Spoof Component Check
For this liveness detection, the algorithm logic will return 3 recommendations for accepting or rejecting a selfie. The spoof component will check the specific defined obvious characteristics of a suspect spoof selfie by defining visual characteristics without giving selfie source information. 
The defined obvious spoof artifacts consist of Nodeflux passive liveness detection has been trained is mentioned below:
Spoof Edge: it’s mostly found on paper and replay attacks which has border characteristic
Deformation: it’s mostly found on a 3D paper mask, cut mask, and 2D paper mask
Artifact: It's mostly found on replay attacks, which gives characteristic moire and reflection. The Moire pattern is a character of obvious pixelated-like pattern that will be found on replay attack from a device screen, LCD, or monitor. Reflection is sometimes found on a replay attack case from a screen photo medium that reflects an image from the device.
Spoof Characteristic
Condition
Example
Border Edge
Obvious Print Attack on Printed paper with border
​
​
​
Border Edge
Border from device
​
​
​
Un-natural deformation
Printed Mask paper with fine cut
​
​
Cut Border
Printed Mask paper with border
​
​
Reflection
Obvious on laptop screen attack contain light reflection and shadow reflection
​
​
Reflection
un-Obvious on laptop screen attack contain light reflection
​
​
Moire
Obvious
​
​
Response API Passive Liveness
The response API passive liveness has JSON structure which is shown below.
Certain Live
{
    "job": {
        "id": "<job_id>",
        "result": {
            "status": "success",
            "analytic_type": "FACE_LIVENESS_V4",
            "result": [
                {
                    "face_liveness": {
                        "doubt": false,
                        "liveness": 0.8125213861465455
                    },
                    "image_quality": {
                        "doubt": true,
                        "contrast": true,
                        "sharpness": true,
                        "brightness": true
                    },
                    "face_size_check": {
                        "face_h": 286,
                        "face_w": 213,
                        "pass_min_face_size": true
                    },
                    "spoof_components": {
                        "artifact": 0.18747861385345457,
                        "spoof_edge": 0.05000110630019394,
                        "deformation": 0.11415836513042454
                    }
                }
            ],
            "code": "25201",
            "desc": "It is likely detect certain live selfie"
        }
    },
    "message": "It is likely detect certain live selfie",
    "ok": true
}
​
Certain Spoof
{
    "job": {
        "id": "<job_id>",
        "result": {
            "status": "success",
            "analytic_type": "FACE_LIVENESS_V4",
            "result": [
                {
                    "face_liveness": {
                        "doubt": false,
                        "liveness": 0.4712049414714178
                    },
                    "image_quality": {
                        "doubt": false,
                        "contrast": true,
                        "sharpness": true,
                        "brightness": true
                    },
                    "face_size_check": {
                        "face_h": 278,
                        "face_w": 214,
                        "pass_min_face_size": true
                    },
                    "spoof_components": {
                        "artifact": 0.46381728291511537,
                        "spoof_edge": 0.5287950585285822,
                        "deformation": 0.16397337168455128
                    }
                }
            ],
            "code": "25202",
            "desc": "It is likely detect certain spoof object"
        }
    },
    "message": "It is likely detect certain spoof object",
    "ok": true
}
Doubt Selfie
{
    "job": {
        "id": "<job_id>",
        "result": {
            "status": "success",
            "analytic_type": "FACE_LIVENESS_V4",
            "result": [
                {
                    "face_liveness": {
                        "doubt": true,
                        "liveness": 0.5377008318901062
                    },
                    "image_quality": {
                        "doubt": true,
                        "contrast": true,
                        "sharpness": true,
                        "brightness": true
                    },
                    "face_size_check": {
                        "face_h": 343,
                        "face_w": 246,
                        "pass_min_face_size": true
                    },
                    "spoof_components": {
                        "artifact": 0.33950707601464314,
                        "spoof_edge": 0.13402243132392566,
                        "deformation": 0.4622991681098938
                    }
                }
            ],
            "code": "25203",
            "desc": "It is likely detect doubt selfie"
        }
    },
    "message": "It is likely detect a doubt selfie",
    "ok": true
}
Our API Response consists of some information:
Liveness score information and doubt message: informing liveness score from range 0.0 to 1.0, this score telling that suggestion decision to reject, or accept a selfie. The doubt binary value (True/False) tells you if the model gets an uncertainty condition to define it as a spoof or live, by this information you can take another decision to recheck the doubt selfie.
Image quality attribute information: the image quality attribute gives you a binary value if the selfie is expected to fulfill the image requirement as “true” and if it is below the image requirement it will give a value of “false”.
Spoof component value. This value gives you explainability about how the algorithm work. We give you information about artifact spoof, deformation, and spoof edge characteristics, so by using this value you can have a second layer decision to check all the spoof signals if there are any doubt cases. Our spoof component threshold is 0.6. If the spoof component score is higher than the threshold, the model tells us that it detects a strong spoof and vice versa.
Here is the best practice about how you can utilize this value for the failover mechanism.
If a spoof component value is greater than the threshold (>0.6), it means the model detects a spoof object signal, so you have to reject the selfie
If a spoof component value is lower than the threshold (<0.5), it means the model detects a low spoof signal, so that you can decide to accept the selfie
If all the spoof values are between 0.6 and 0.5, which is near to the threshold or not strong enough, it means the model is in uncertain condition, you need to check the selfie manually before deciding to accept or reject
Defining Liveness Threshold
We suggest 3 levels of threshold on general liveness score (on parameter liveness on the API) to define the liveness detection decision for rejecting or accepting a selfie. 
"face_liveness": {
                "doubt": false,
                "liveness": 0.006455930027489861
            }
We classify the decision into 3 conditions:
Strong spoof: it is a case that you must reject
Doubt Case: case when a spoof component is on the grey area (below the first threshold) or when the image quality assessment results in doubt value is "true". This doubt condition tells that the model has uncertainty to decide the selfie is a spoof or live so the human check still needs on this process to ensure the selfie.
Strong live: a case that you can accept the selfie, the model detects all low spoof signal which the value of all spoof components are less than 0.6.
Threshold Range for general liveness score
Inspecting Spoof Case Through Spoof Component Information
On this Liveness model, you can inspect how the model works and where the model is failing. We give you the spoof component score of each model to help you make a decision on tailoring the failover mechanism. Here are the spoof component and the value which will explain to you how the model is working.
"spoof_components": {
                "artifact": 0.0844744449853896,
                "deformation": 0.946455930027489861,
                "spoof_edge": 0.0648913257879514
            }
Spoof edge detection: if the value is above the threshold (>0.6), the model detects a strong spoof edge
Un-natural Deformation check: if the value is above the threshold (>0.6), the model detects a strong deformation characteristic
General artifact check: if the value is above the threshold (>0.6), the model detects strong general spoof artifacts.
​
​