Sentiment analysis, also known as opinion mining, automatically analyzes user comments to determine whether a piece of writing expresses a positive or negative sentiment. To do this, sentiment analysis uses text mining methods. Text mining is the automatic analysis of text written in natural language.
And this is precisely where the first problem arises: Natural language is not made up of lists of positive and negative words. Simple analytical methods search the text for words with a positive or negative meaning according to a lexicon created in advance, tailored to the specific topic. While this method provides a very rough overview, it’s not very useful for capturing the actual sentiment. When analyzing a subjective review of a product, even the frequency of supposedly positive or negative words is not very informative.
Consider these two reviews: “I’m thrilled” and “Very good, it fulfills its purpose.” The first sentence contains a positive word, “thrilled,” the second sentence contains two positive words, “good” and “fulfills.” Simple statistical analysis would give the second sentence a higher score, whereas humans would consider the review mediocre or negative. Accordingly, successful sentiment analysis needs artificial intelligence.
Another issue is that people on social media in particular express their opinions and converse as they would to a friend. They don’t always strictly follow the rules of English grammar. Many sentences have a completely different meaning in their broader context. Recognizing these nuances is a major challenge for analysis tools. Furthermore, a lot of language, especially teenage slang, is heavily influenced by short-lived trends.
As a result, a sentiment analysis tool needs precise knowledge of the target audience and the product environment. This is where machine learning methods can be applied to gradually train the tools and improve the quality of results over time.