mAuth User Study: Art Attribution Authoritativeness Evaluation Results

31 users participated in a remote task-based evaluation of the mAuth web application for assessing authoritativeness in art history attributions. The study measured User Satisfaction, Rank Satisfaction Score, and Perception of Authoritativeness Score across three common connoisseurship scenarios using a 5-point Likert scale. Results include inter-rater agreement calculated via Fleiss Kappa and user feedback on dimensions affecting the ranking model.

Use Cases

• Evaluate ranking model performance for art attribution based on User Satisfaction (US), Rank Satisfaction Score (RSS), and Perception of Authoritativeness Score (PAS) measures.
• Analyze user feedback on dimensions affecting authoritativeness ranking in art historical data.
• Study inter-rater agreement in subjective evaluations of information quality using the Fleiss Kappa measure.
• Compare user satisfaction across different information availability scenarios (unanimous, debated-undocumented, debated-well-documented).

Strengths

• Includes results from 31 users, providing a basis for statistical analysis.
• Measures three specific evaluation metrics (US, RSS, PAS) with defined Likert-scale questions.
• Calculates inter-rater agreement using the Fleiss Kappa measure, adding methodological rigor.
• Covers three distinct connoisseurship scenarios, allowing for comparative analysis.

Limitations

• Row count and specific data columns are unknown, which limits suitability assessment.
• Column-level documentation is absent; field semantics must be inferred after download.
• Last update date is unknown; freshness unverified.

Provenance

Source

Marilena Daquino, via paperswithcode platform.

Collection Method

Data collected via a remote user study using a web application and an evaluation form.