Automating Students & Employees Assessment Process with AI

The Challenge We Wanted to Address

Learning standards have got a new hold these days! But the education industry stakeholders' reactions are not aligned with the much-required transition movements. A gap has been built and is continuing to widen with issues such as data protection, outdated certification processes as well as data accessibility and archiving. There was a high need for the solution to bring in a positive technical revolution in the education sector.

Our Product Idea

We decided to work on a question bank, coupled with the AI and ML technology that could be trained to assess the various factors that generally the interviewer observes while meeting the interviewee.

The questions bank we create needed to have both valid questions along with relevant answers. Moreover, we wanted the final tool to assess the interviewee using natural language processing.

The Ultimate Features of the Product

In the final tool, you can pick and upload relevant articles. The software will create relevant questions out to the content. Then, ask these questions to the candidate or viewers.

Then, the tool will take input of interviewee voice and judge the answers in semantic sense using Natural Language Processing. Moreover, it will also assess the quiver in voice and overall confidence of the candidate.

The AI-based software also takes video feed to check body language and facial expressions during the interview and uses it to create an assessment of the user’s emotional state during the interview.

Our Solution

To create the perfect assessment software using AI and ml technology we created a work flow. In this we first focused on voice, threshold value for selection then database work and stamps and finally emotional detection.

Running similarity check through natural language processing

For this we used cosine similarity with ###########WORD TO VECTOR APPROACH to check (################SEMANTIC SIMILARITY BETWEEN ACCURATE ANSWER AND USER’S ANSWER) average find function (##########AVERAGE FUNCTION after SIMILARITY PERCENTAGE FRO assigning score card on the basis of answer). In this process the use of natural language processing helps the computer determine if the content of interviewee is relevant to the question on the basis of semantics. It does not need word to word response. Instead, it will check the users’ answers in comparison with the content to determine it is right and to what extent. There is a threshold value in place and it the interviewee answer matches or exceeds that percentage, then it is good to go.

Voice bot

In this we used function and classes (#######OOP’s concept) along with Google (#######OOP’s concept) libraries for text to voice and voice to text. After this, we integrated voice bot into the front end.

Database and Timestamps

For average connect with MySQL for storing conversations and time stamps

Emotion detection

We identified 9 major emotions data sets and started with preprocessing labeling and sitting it into a pre-trained model using OpenCV for detecting my emotions in live feed.

Finally we integrated all three and started live-testing after front-end deployment.

Our Biggest Challenge

Finding the right pre-trained model for emotion detection

In the case of both emotions and voice sentiment detection, finding the right model was close to impossible due to the lack of unbiased models.

Solution: to overcome the issue, we worked on a custom model that could identify emotions and voice sentiments to assess the interviewee’s state of mind while giving the answers.

The Final Result

At the end of the project, we created a software that fully automated the process of taking interviews, from asking questions, to checking answers and running a complete sentiment analysis based on live feed and voice modulation to determine the level of confidence in the candidate.

It can work well for both offices and educational institutions.