A practical user feedback classifier for software quality characteristics

It is common practice for users to provide feedback on apps through social media or app store reviews. This feedback is a rich source of requirements for these apps. However, manually analyzing vast amounts of user feedback is unfeasible, so automated user feedback classifiers are useful tools. This research work presents a user feedback classifier based on Machine Learning (ML) for the classification of reviews according to software quality characteristics complaint with the ISO25010 standard. We developed this approach by testing several ML algorithms, features, and class balancing techniques for classifying user feedback on a data set of 1500 reviews. The maximum F1 and F2 scores obtained were 60% and 73%, with recall as high as 94%. This approach does not replace human specialists, but reduces the effort required for requirements elicitation.