A dataset containing coding from a study author and four reliability coders for a research project. Each coder worked with 10% of the total dataset, and their work was individually compared with the author's. Inter-rater agreement and Cohen's Kappa scores were computed for all author-coder pairs using R version 3.6.1 and the irr package.
Use Cases
- Calculate inter-rater agreement metrics based on the described author-coder comparisons.
- Replicate reliability analysis using the provided R script.
- Study methodological approaches for assessing coding consistency in qualitative or quantitative research.
- Train students on computing and interpreting Cohen's Kappa and other reliability statistics.
Strengths
- Includes comparisons between a study author and four independent coders, providing multiple reliability assessments.
- Provides specific computational details: R version 3.6.1 and the irr package version 0.84.1 were used.
- An R script is attached for replication purposes.
Limitations
- Description metadata is limited; actual data quality requires manual inspection after download.
- Column-level documentation is absent; field semantics must be inferred after download.
- Row count is unknown, which may limit suitability assessment.
Provenance
- Source
- Kate Mesh, with coders from The University of Texas at Austin and the University of Haifa.
- Collection Method
- Manual coding by research assistants, with statistical comparison performed in R.
- Time Range
- null
- Freshness
- Last updated 2024-03-18 09:23:50; freshness should be verified.
- Geography
- null