Here is a report on a cloud computing project, Humanoid for HR, Sentiment Analysis of the interview candidate using AWS AI /ML services by Sachin Suvarna, Ganesh Pai, and Vikas Arora.
Objective
Employees are a valuable resource for any organization. This makes the evaluation process of the employee even more important to handle for an organization. Interviewing the candidate is the most important part of the evaluation process of the candidate. Our tool assists the HR team in this aspect of the evaluation process. This tool aims to provide an unbiased opinion about the candidate during a Human Resource interview. It proposes to provide a solution to human errors that the interviewer may encounter while judging the candidate during an HR interview.
Problem statement
In the current business situation, every business activity is getting inclined towards the virtual environment, every department of the organization are moving towards virtualization, to reap its benefits. The HR department is no exception to this, most of the activities of this department are getting virtual too and interviews are one among them. As these new business scenarios bring advantages and opportunities to every business, they too bring with them new challenges and risks.
Virtual interview too brings its own advantages and challenges. No physical presence of the interviewee in the interview process may increase human errors in the evaluation. Interviews could be faked by the interviewee, instances have been found where someone else speaks on the behalf of the interviewee or the candidate just lip synchs and the questions are answered by a person who is not on the camera.
Solution
Our solution attempts to resolve the problems or challenges posed by virtual interviews. The interviewee would be recorded during his interview. Our tool Humanoid for HR will analyze the recorded video interview and will provide an analysis of the interviewee based on his facial reactions and verbal communications done by him during the interview using AWS tools and AI/ML services.
The analysis of the interview happens in two phases, Video analysis wherein all the facial expressions of the candidate are detected, compared and analyzed on different parameters using AWS Rekognition and Comprehend. Secondly, Audio analysis is done and sentiment analysis is performed on the spoken words using AWS Transcribe and Comprehend. This data is then used for comparative analysis.
The consolidated data of the analysis of the Audio and Video is then used to provide an evaluation of the candidate in the form of a report. The tool also attempts to highlight any suspicious activity of faking the interview.
Check out how sentiment analysis using python can be carried out.
Role of Cloud Services
AWS Cloud Platform: AWS has the broadest and deepest set of machine learning and AI services for use. One can choose from pre-trained AI services for computer vision, language, recommendations, and forecasting.
AWS Lambda: AWS Lambda is used for all the processing code in the application. This serverless architecture is used to take advantage of paying only for the compute time.
AWS Rekognition: AWS Rekognition is a cloud-based Software as a Service (SaaS) computer vision platform. AWS Rekognition makes it easy to do video analysis in the applications using proven, highly scalable, deep learning technology that requires no machine learning expertise to use. With Amazon Rekognition, one can identify objects, people, text, scenes, and activities in images and videos. Amazon Rekognition also provides highly accurate facial analysis and facial search capabilities that one can use for different use cases.
AWS Transcribe: Amazon Transcribe is a service from AWS that makes it easy for developers to add speech-to-text capability to their applications. Amazon Transcribe uses a deep learning process called automatic speech recognition (ASR) to convert speech to text quickly and accurately.
AWS Comprehend: Amazon Comprehend is a natural language processing (NLP) service from Amazon that uses machine learning to find insights and relationships in a text. Machine learning is particularly good at accurately identifying specific items of interest inside vast swathes of text and can learn the sentiment hidden inside language at almost limitless scale.
AWS Simple Storage Service (S3): AWS S3 service is been used to avail highly scalable storage service to store and process Video files.
AWS Glue: AWS Glue is a fully managed extract, transform, and load (ETL) service that makes it easy for us to prepare and load the data for analytics. One simply point AWS Glue to the data stored on AWS, and AWS Glue discovers once data and stores the associated metadata (e.g. table definition and schema) in the AWS Glue Data Catalog. Once catalogued, the data is immediately searchable, queryable, and available for ETL.
Crawler: One can use a crawler to populate the AWS Glue Data Catalog with tables. This is the primary method used by most AWS Glue users. A crawler can crawl multiple data stores in a single run. Upon completion, the crawler creates or updates one or more tables in the Data Catalog. Extract, transform, and load (ETL) jobs that one define in AWS Glue use these Data Catalog tables as sources and targets. The ETL job reads from and writes to the data stores that are specified in the source and target Data Catalog tables.
Amazon Athena: Amazon Athena is an interactive query service that makes it easy to analyze data in Amazon S3 using standard SQL. Athena is serverless, so there is no infrastructure to manage, and one can pay only for the queries that one run. Athena is easy to use, simply point to the data in Amazon S3, define the schema, and start querying using standard SQL. Most results are delivered within seconds. This makes it easy for anyone with SQL skills to quickly analyze large-scale datasets.
Amazon QuickSight: Amazon QuickSight is a fast, cloud-powered business intelligence service that makes it easy to deliver insights to everyone in the organization. As a fully managed service, QuickSight lets one easily create and publish interactive dashboards that include ML Insights. Dashboards can then be accessed from any device and embedded into your applications, portals, and websites.
AWS Simple Queue Service (SQS): AWS SQS is a fully managed message queuing service from AWS, that enables to decouple and scale microservices, distributed systems, and serverless applications.
AWS Simple Notification Service (SNS):- AWS SNS is used by publisher systems to fan-out messages to a large number of subscriber endpoints for parallel processing, including Amazon SQS queues, AWS Lambda functions.
Business flow
- Our application starts with the drop of the interview video of the candidate in the S3 bucket which triggers respective Lambda functions sequentially to process the interview video and perform analysis over the same
- The analysis of the video happens in two phases, a) the visual facial emotions are captured and analyzed using AWS Recokgnition and Comprehend b) the spoken words of the candidate are been analyzed and emotions are captured from his spoken language with the help of AWS Transcribe and AWS Comprehend
- The analysis of the video happens in different phases in different lambda functions as the data is moved across these functions through S3 buckets in json format.
- The final analyzed data is stored in S3 is then moved through AWS Glue and Crawler to Athena to query the analyzed data
- Finally, the queried data is then presented in graphical format using AWS QuickSight in the form of pivot tables and Bar charts
The following points are worthy to be mentioned related to the tool:
- Real-world impact – This concept of this tool is one of a revolutionary idea in the Human resources industry, which should greatly assist people in taking unbiased decision about the candidates
- Innovation – Innovative technologies are used in the development of this tool like Artificial Intelligence and Machine Learning
- Main achievements – The important goal in the development of this tool is to learn the features of AWS services and usage of AI and ML in Cloud Computing
About the Authors
- Sachin Suvarna: Associate, Cloud Operations, Nomura Services India
- Ganesh Pai: Associate, Global Technologies Platform, Nomura Services India
- Vikas Arora: Vice President, Global Technologies Platform, Nomura Services India
