Value-Added Model Sensitivity Analysis Across 18 Education Datasets

This dataset supports a study examining the sensitivity of value-added model (VAM) estimates to different test scoring methods. The analysis uses empirical item response data from 18 distinct education datasets to compare estimates derived from methods like mean scores versus item response theory scores. It finds that scoring choices can lead to variations of over 20 percentile points in teacher or school rankings.

Use Cases

• Analyze the correlation between value-added estimates from different scoring methods across the 18 education datasets.
• Compare the classification stability of teachers or schools into performance quartiles based on varying scoring approaches.
• Investigate the relationship between the completeness of item response data and the dispersion in value-added percentile ranks.
• Examine the sensitivity of school-level value-added estimates to the underlying test scoring method.

Strengths

• Analysis is grounded in empirical item response data from 18 distinct education datasets.
• Findings are based on comparing multiple scoring methods (e.g., mean scores vs. item response theory scores) on the same student and item data.
• The study quantifies impact, reporting variations in value-added percentile ranks of over 20 points.

Limitations

• The specific item response data from the 18 source datasets is not directly accessible; the provided data is for replication of the study's analysis.
• The dataset's scope is defined by the 18 source studies, which may introduce selection bias or limit generalizability.
• The raw data structure, columns, and sample size for replication are not described in the input.

Provenance

Source

Josh Gilbert Dataverse

Collection Method

Replication data for a published study analyzing 18 existing education datasets.

Freshness

Last updated on 2026-02-05.