Annual: 2019

AP021 »
Expression Extraction For Lie Detection using image processing.
📁Machine Learning
👤Narayan Raval D
 (LD college of engineering)
📅Oct 09, 2019
Regional Final

👀 4837   💬 75

AP021 » Expression Extraction For Lie Detection using image processing.


Lie detection is an evolving subject. Polygraph techniques is the most trending so far,but a physical contact has to be maintained.The project proposes the lie detection by extracting facial expressions using image processing. The captured images to be analyzed is broken into facial parts like eyes, eyebrows,nose etc. Each facial parts is then studied to determine various emotions like eyebrows raised and pulled together,raised upper eyelids,lips stretched horizontally back to ears signifies fear while eyebrows down and together, narrowing of the lip shows anger. All the emotions can be aggregated to determine wheather a person is lying or not. The interrogation video or live video is broke down into various facial images of the particular individual. Different emotions from the various images is collected and processed with the general face reading criteria to evaluate his truthfullness.

Demo Video

  • URL:

  • Project Proposal

    1. High-level Project Description


    Polygraph technique has been the most successful tecnique so far for detecting truthfullness of the person under test. But a physical contact has to be maintained for better efficiency which is a major drawback of this techniqe. Polygraph is a wired system which instigates and panic and anxiety to the person under test which triggers false positive results. 

    Due to rapid evolution in computer vision and artificial intelligence we proposes a system which extracts face expressions using image processing to determine wheather a perso is lying or not.

    2. Purpose of the design

    The main purpose of our design is to create a technique which doesn't involve any physical contact to avoid any false positive result. The research makes use of image processing techniques to extract facial expressions and with the given algorithm it verifies the person's emotions. All the emotions are summed up to determine his/her truthfullness.

    3. Application scope

    • Today there are different ways to beat a polygraph test and many criminal suspects rely on it to free themselves as it is even considered as one material of evidence. Our system will proves advantageous in those situations as it deals with the camera in the interrogation room. With the help of image processing we segments the video into different images and each image is analyzed and the emotions are recorded. All the emotions are fed into a fuzzy logic algorithm which determine wheather the person is lying or not. The criminal suspect cannot beat the system which records micro details of his emotions. FPGAs proves to be a great success because of its high computational speed in real time.
    • It can be used even in an past recorded video to analyze the emotions.
    • It can be used as a trial to determine the efficiency.

    4.Targeted users

    • Criminal suspects.
    • Analyzing a past recorded video.
    • As a trial by any users.
    • It can be used in students to determine thier depression level.

    5. Why FPGA?

    • FPGA's Cyclone V provides 13,917k bits embedded memory which provides enough memory to save many recorded videos.
    • UART to USB connectors by which we can connect HDMI cables as well.
    • 12 bit resolution in arduino expansion boards to infiltrate deep image size.


    2. Block Diagram

    First of all facial expressions has to be evaluated to initialise the project. The facial expression extraction can be divided into three main categories:- 

    1. Face detection
    2. Facial feature extraction
    3. Emotion classification and lie detection

    After all the above process the resulting emotions are recorded to determine the lie.

    1. Face detection

    There are various face detection approaches so far.

    • Knowledge based approach : Knowledge based approaches are based on the rules derived from knowledge on face geometry. Defining the rules is based on relative distances and position of facial features.
    • Feature invariant approach :  Facial features are detected and grouped according to geometry of face in this approach.
    • Template based approach : In template based approach standard pattern of human face is used as the base.
    • Appearance based approach : It considers the human face in terms of a pattern with pixel intensities.

    We will be using feature invariant approach fpr face detection.

    2. Feature extraction 

    • Face detected in previous stage is analyzed to identify eye,eyebrows & mouth regions.
    • Y co-ordinate of the eye is identified with the use of horizontal projection.
    • Then areas around Y coordinates are processed to identify the exact location of the feature.
    • Finally a corner point algorithm is used to obtain the required corner point from the feature regions.

    3.Emotion classification and lie detection

    All the emotions are fed in a fuzzy algorithm and recorded to classify various emotions. For example eyebrows raised, eyes widened, mouth open etc determines surprise while eyebrows down & together , eyes glare and narrowing of the lips shows anger. In this way we can segment out various emotions like contempt, guilt,anger,sad and shock. All the emotions are collectively processed to determine the truthfullness.

    3. Intel FPGA Virtues in Your Project

    Humans expresses various emotions like sad,guilt,surprise etc while talking. Our research focuses on extracting those distinct emotions while they speak which requires high computational speed. The efficiency of the project depends mainly on :-

    • High computational Speed.
    • Better performance.
    • Low latency as possible.
    • It should be able to process any image sizes extracted from the video.

    Our project uses fpga because of its folowing virtues:-

    • FPGAs are capable of parallel processing which makes it faster comaparatively.
    • We are using a DA algorithm which is possible in FPGAs to improve its performance.
    • Since FPGAs do not depend on any operating system and communication does not have to be passed through buses, its latency is low of about 1 microsecond or less depending upon the applications.
    • Since FPGAs are able to use temporal parallelism and using filters for a high image size bit is often less useful in software based techniques of image processing. It requires a real time hardware system and that's where FPGA fit right in the place.

    All the above reasons made it valid that for a real time image processing for lie detection , FPGA boards is our first choice.

    4. Design Introduction

    1. Introduction

    Humans have a well defined rigid skull structure and therefore they can perform finite number of facial expressions.Our proposed system extract these expressions ,classifies them and categorize them to determine the respective emotions.

    With respect to the interrogation questions, one can expect a certain emotions from the person under test. Each respective emotions can be categorized to be more lie or more truth. If the peak value is pointing towards lie, then the person can be assumed to be lying.

    2. Literature

    Facial Expression Recognition ususally performed in four-stages consisting of pre-processing, face detection, feature extraction and expression classification.


    In this project we applied various deep learning methods (convolutional neural networks) to identify the key seven human emotions: anger,disgust,fear,happiness,sadness,surprise and neutrality.

    There are different approaches for Facial Expression Recognition, we are using the Neural Network Approach.

    • Neural Network Approach                                                                                                             The neural network contained a hidden layer with neurons. The approach is based on the assumption that a neutral face image corresponding to each image is available to the system. Each Neural network is trained independently with the use of on-line back propagation. 


    5. Function Description

    1. Configuiring OPEN VINO toolkit                                                                                   

     1. Configure Model Optimizer for the framework.                                                                             

     2. Convert a trained model to produce an optimised intermediate representation of the model based on the trained network topology, weights and biases value.                                                     

     3. Test the model in the Intermediate Representation format using the inference engine in the target environment by the validation application.                                                                                 

     4. Integrate the Inference engine to deploy the model.

    2. Model Optimizer

    Model Optimizer produces an OpenVINO supported framework as a trained model input and an Intermediate Representation of the network as output.

    Model Optimizer has two main purposes:

    1) Produce a valid Intermediate Representation.

    2)Produce an optimized Intermediate Representation.

    3. Inference Engine

    After an Intermediate Representation is created by the Model Optimizer input data can be inferred by the Inference Engine.

    The Inference engine is a C++ library with a set of C++ classes to infer input data and get an result.

    6. Performance Parameters

    • Open Stater Kit has its own inference engine which decreases the computational speed.
    • Expected latency was 1 microsecond and practicl latency was found to be around 2 microsecond.
    • It supports OpenCv which is helpful for certain library packages.

    7. Design Architecture

    1. The input video is fed to the OpenVINO stater kit which thereby display the result.

    2. Collection of images are segmented and face is detected from the image.

    3.Facial expressions are detected by landmarking facial regions using Deep Neural Network(DNN).

    4. The recognized is fed to the classifier to determine the percentage of lie or truth.


    🕒 Jul 08, 2019 01:53 AM
    interesting approach good work!
    🕒 Jul 08, 2019 01:49 AM
    nice work.......kishanbhai!
    🕒 Jul 08, 2019 01:45 AM
    good job!
    🕒 Jul 08, 2019 01:40 AM
    nice project............surya!
    🕒 Jul 08, 2019 01:28 AM
    very good work bachche log!
    🕒 Jul 08, 2019 01:24 AM
    nice project
    🕒 Jul 07, 2019 10:11 PM
    aditya patel
    nice project
    🕒 Jul 07, 2019 09:39 PM
    nice one
    🕒 Jul 07, 2019 06:16 PM
    Rajvirsinh Mahavirsinh Chudasama
    Nice project
    🕒 Jul 07, 2019 05:49 PM
    nice one
    🕒 Jul 07, 2019 03:52 AM
    best of luck
    🕒 Jul 07, 2019 03:51 AM
    good idea
    🕒 Jul 07, 2019 03:50 AM
    nice project
    🕒 Jul 07, 2019 03:47 AM
    Alpesh Patel
    Good project
    🕒 Jul 07, 2019 03:44 AM
    excellent idea
    🕒 Jul 07, 2019 03:27 AM
    wish you luck guyz
    🕒 Jul 07, 2019 03:24 AM
    hope you guyz go ahead in this
    🕒 Jul 07, 2019 03:22 AM
    wish you luck guyz
    🕒 Jul 07, 2019 03:20 AM
    nice work guyz
    🕒 Jul 07, 2019 03:19 AM
    Megh Parikh
    Best of luck Narayan
    🕒 Jul 07, 2019 02:41 AM
    Excellent project.. Surya ( best of luck).
    For this project.
    🕒 Jul 07, 2019 02:18 AM
    Dhruv shukla
    Excellent work narayan.
    🕒 Jul 07, 2019 12:56 AM
    Gohil Jayeshkumar
    Excellent work
    🕒 Jul 07, 2019 12:14 AM
    Amazing project
    🕒 Jul 07, 2019 12:04 AM
    🕒 Jul 07, 2019 12:02 AM
    Very nice project good work
    🕒 Jul 07, 2019 12:00 AM
    Akash nair
    Nice work
    🕒 Jul 06, 2019 11:57 PM
    Amazing work
    🕒 Jul 06, 2019 11:55 PM
    excellent project
    🕒 Jul 06, 2019 10:48 PM
    divya r
    🕒 Jul 06, 2019 06:32 PM
    saffin yadav
    Best of luck
    🕒 Jul 06, 2019 06:28 PM
    nice work guys
    🕒 Jul 06, 2019 06:20 PM
    saffin yadav
    God job beta
    🕒 Jul 06, 2019 06:18 PM
    Best of luck dude
    🕒 Jul 06, 2019 04:52 AM
    Ajay Kukadiya
    🕒 Jul 05, 2019 11:26 PM
    Thank you sir!
    🕒 Jul 06, 2019 02:10 AM
    Amish Joshi
    Liers gonna hate this!!.. Best of luck guys
    🕒 Jul 05, 2019 02:07 AM
    it's so nice of you. thank you
    🕒 Jul 06, 2019 02:11 AM
    Nipa raval
    Nice one
    🕒 Jul 05, 2019 01:31 AM
    Thank you madam
    🕒 Jul 06, 2019 02:11 AM
    Smoker rules
    Nice idea
    🕒 Jul 04, 2019 02:06 PM
    thank you sir
    🕒 Jul 06, 2019 02:12 AM
    Rajesh shah
    Good work
    🕒 Jul 04, 2019 01:59 PM
    Thank you sir
    🕒 Jul 06, 2019 02:12 AM
    Naitik jp
    Excellent project....all the best
    🕒 Jul 04, 2019 03:54 AM
    Thank you sir
    🕒 Jul 06, 2019 02:12 AM
    Excellent Project
    🕒 Jul 04, 2019 03:48 AM
    thank you sir
    🕒 Jul 06, 2019 02:13 AM
    Rathwa Parvati N.
    mind blowing project
    🕒 Jul 03, 2019 11:52 PM
    Thank you madam
    🕒 Jul 06, 2019 02:13 AM
    Harsh parmar
    Interesting project . Keep it up
    🕒 Jul 03, 2019 01:37 PM
    Thank you sir!
    🕒 Jul 03, 2019 07:49 PM
    dharmendra raval
    nice idea keep it up
    🕒 Jul 03, 2019 03:59 AM
    Thank you sir
    🕒 Jul 03, 2019 07:50 PM
    Shalini nai
    Excellent concept
    🕒 Jul 02, 2019 11:53 PM
    Thank you madam
    🕒 Jul 03, 2019 07:50 PM
    Jigar parmar
    Excellent work
    🕒 Jul 02, 2019 10:17 PM
    Thanks bro
    🕒 Jul 02, 2019 10:25 PM
    🕒 Jul 02, 2019 09:47 PM
    Thank you bro!
    🕒 Jul 02, 2019 10:03 PM
    Excellent project
    🕒 Jul 02, 2019 09:24 PM
    Thank you sir
    🕒 Jul 02, 2019 09:27 PM
    This is very nice. Good job.
    🕒 Jul 02, 2019 02:17 PM
    Thank you madam!
    🕒 Jul 02, 2019 09:00 PM
    Rekhaben rathava
    Excellent idea.looking forward for this project. Best of luck
    🕒 Jul 02, 2019 11:15 AM
    Thank you madam
    🕒 Jul 02, 2019 11:20 AM
    This is an interesting project! But why don't you use a voice?
    🕒 Jul 02, 2019 04:13 AM
    We have kept voice feature as an extension for this project. Thanks for ur attention to our project.
    🕒 Jul 02, 2019 11:21 AM
    AP021 🗸
    Intel's OpenVino stater kit. Thanks for your attention to our project.
    🕒 Jun 30, 2019 10:05 PM
    Doreen Liu
    Please complete the rest part of the proposal. The deadline of the first stage is 2019-06-30.
    🕒 Jun 26, 2019 12:00 PM
    narayan raval
    sir i just unable to upload my block diagram which is all ready even description, what should i do for that sir.
    🕒 Jul 01, 2019 03:05 AM
    Doreen Liu
    Plese try again? Our people has fixed the webpage today morning.
    🕒 Jul 01, 2019 10:41 AM
    Doreen Liu
    Which FPGA boar will you use?
    🕒 Jun 26, 2019 12:00 PM
    open vino starter kit
    🕒 Jul 01, 2019 03:10 AM

    Please login to post a comment.