Amazon Product Reviews Annotated for Complexity via Appraisal Theory

Performance Evaluation using DistilRoBERTa Model is a manually annotated dataset of Amazon product reviews. The reviews are annotated according to perceived complexity using the attitudinal categories of appraisal theory: appreciation and judgment. The dataset was created by Shoukat Ullah and last updated on 2026-05-26.

Use Cases

• Training hybrid deep learning models for sentiment analysis based on appraisal theory categories.
• Evaluating product usability and learning curve based on 'appreciation' annotations.
• Assessing product effectiveness and appropriateness based on 'judgment' annotations.
• Benchmarking transformer models like DistilRoBERTa against other state-of-the-art approaches.
• Applying explainable AI (XAI) methods like SHAP to understand model predictions for product feedback.

Strengths

• Dataset is manually annotated according to a theoretical framework (appraisal theory).
• The proposed model described achieved a mean fold accuracy of 96.13%.
• The work incorporates explainable AI (XAI) methods for model transparency.

Limitations

• Dataset size is very small at 5.5 KB, indicating limited scope.
• Row count and column-level documentation are unknown.
• Freshness should be verified as the last update date is in the future (2026-05-26).

Provenance

Source

Amazon product reviews, annotated by Shoukat Ullah.

Collection Method

Manually annotated according to appraisal theory categories, then cross-validated via SHAP.

Freshness

Last updated 2026-05-26 17:43:30