Creating a Sentiment Analysis Solution Based on Customer Feedback Processing

One question shaped every decision here: can a model score a review the way the person who wrote it would have? If the answer is yes, then every piece of unstructured data in the archive turns into a sentiment score, whether or not a star was ever attached to it.

The rating lived on the storefront. The complaint lived in the text, inside a support ticket or a comment nobody exported. Our sentiment analysis model predicts the missing number and puts both halves of the picture in one table, which is where opinion mining stops being an academic exercise and starts being a merchandising tool.

Our machine learning development services included fine-tuning BERT for regression using one million Amazon reviews paired with their ratings. Five-class sentiment classification would have forced each review into a bucket, and a 3.6 rounds away into something less useful. Regression keeps the output continuous. Sorting every comment into positive, negative, or neutral would have answered a smaller question than the one the client asked. The dataset split 850,000 for training, 75,000 for validation, and 75,000 held back for a final test nobody touched until the end.

Baseline came first. A rule-based sentiment analysis lexicon paired with TF-IDF and logistic regression gave a mean absolute error of 0.71 stars, which set the bar for the BERT-based review analysis that followed. The fine-tuned model landed at 0.38, roughly 46% less error. Sequence length caps at 256 tokens, enough for about 96% of reviews without truncation. Three epochs, batch size 32, and the full training run finished in under nine hours.

Accurate sentiment analysis on a 1-5 scale is hardest in the middle. Exact-star agreement runs near 82% at the ends of the scale and drops to 58% on 3-star text, where neutral sentiment and mild disappointment use almost the same words. The sentiment expressed in “it’s fine I guess” sits close to 3.2, and no rule set lands that twice in a row. Machine learning techniques that treat this as a five-way choice hide that weakness. Regression exposes it, so the team could see where the model wobbled and weight the training data accordingly.

Nothing about the aspect layer is generic. The 12 categories came out of the client’s own vocabulary, because “runs small” means something precise in apparel and nothing at all in electronics. Two of our data scientists sat with two category managers and hand-labeled the first 4,000 examples before any fine-grained sentiment analysis went near production. Two types of sentiment analysis run side by side in production now: one overall sentiment score per review, and aspect-level scores underneath it.