Sentiment Analysis using Python

Sentiment analysis is the contextual mining of text that identifies and extracts subjective information in input sources to get meaningful insights on which business decisions can be made. It can help a business to understand the social sentiment of their brand, product, or service while monitoring online conversations.

Usually, sentiment analysis maps the responses into three buckets, i.e., positive (My experience has been fantastic), neutral (My experience is just ok), and negative (My experience has been pathetic). Sentiment analysis is the basic tool to detect customer sentiment from social data, gauge brand reputation, and understand customer thought processes.

There are many types of sentiment analysis. Fine-grained sentiment analysis divides the sentiments into fine-grained classes. You can define your classes as Very happy, happy, ok, not happy, not at all happy. They are usually mapped to stars, i.e., very happy is mapped to 5 stars and not happy maps to 1 star. This is usually done for product ratings. Another type of sentiment analysis is emotion detection. Here we primarily detect emotions, like happiness, frustration, anger, sadness, and so on. Many emotion detection systems use lexicons (i.e., lists of words and the emotions they convey) or complex machine learning algorithms. But this type of sentimental analysis depends on culture since different cultures use the same set of words to express different emotions. For example, in India, a phrase like “you are killing me badass” means the customer is highly unsatisfied and angry. In contrast, a section of US people uses the same phrase to express happiness. Aspect-based sentiment analysis focuses on which aspects or features in each response people are mentioning in a positive, neutral, or negative way instead of focusing on the entire review. Another class of sentiment analysis is multilingual sentiment analysis. These days, most youth mix 2 or three languages (and regional dialects or colloquial phrases) to express their views. Here you must identify parts of the text of each language, translate it semantically in one language and then run the algorithm on the resultant text. It is very much computationally intensive.

There are many advantages of sentiment analysis. The most obvious benefit is sentiment analysis sorts data at scale. A typical raw data set for sentiment analysis may consist of 1000s of sentences. IT would be difficult to analyze such a massive data set manually. So, sentiment analysis helps in processing the large data set efficiently and cost-effectively. The other obvious benefit of sentiment analysis is that it works in real-time mode. For instance, several tweets of an angry customer can be detected and assigned to a customer care representative to resolve the aggravated customer. The key strength of any sentiment analysis is its consistency in applying the same algorithm to each input irrespective of the fact that individual tweets are highly subjective, influenced by personal experiences, thoughts, and beliefs. This directly results in improving accuracy and gaining better insights. 

Sentiment analysis can be done using any programming language and specialized library packages. For Python, NLTK, TextBlob, and Spacy are two libraries that are commonly used for sentiment analysis. NLTK is a near-perfect library for all sorts of NLP tasks. TextBlob is based on NLTK and Pattern and is super-specialized for text analytics with features like sentiment analysis, pos-tagging, noun phrase extraction, etc. Spacy is a more advanced and powerful library for performing NLP tasks. 

TextBlob exposes a simple API to access its methods and perform basic NLP tasks. In TextBlob, the text is treated as python strings. So, all string operations can be applied to it. Textblob has easy-to-use functions for tokenization, noun phrase extraction, part of speech tagging, porting and lemming, N-gram analysis, sentiment analysis, spelling correction,  summarizing a large text, and language detection and translation. 

The two most common use case of sentiment analysis is found in twitter analysis and amazon product reviews. Twitter is the most popular microblogging platform that allows businesses to engage personally with consumers. Since it is a free and most popular platform, an ocean of data makes it hard for brands to prioritize which tweets or mentions to respond to first. Twitter sentiment analysis is a tool that automatically monitors emotions in conversations on social media platforms and helps companies respond appropriately.

Amazon Product review is another popular application of sentiment analysis. Amazon is USA-based MNC that focuses on e-commerce, cloud computing, digital streaming, and artificial intelligence products. Its e-commerce platform is one of the biggest online shopping platforms in most countries of the world today. Amazon has a very large customer base that generates tons of data, most important product reviews. So, sentiment analysis over product reviews generates actionable insights for product managers to improve the product, change its pricing, or the way it is promoted.