Human vs AI Agent Code Review Conversations from 300 GitHub Repositories

AgentReviewChat contains 278,790 inline code review conversations from 54,330 pull requests across 300 popular open-source GitHub repositories. It captures interactions between human developers and 16 identified AI agents, enabling analysis of review feedback quality and interaction dynamics. The dataset was created by Suzhen and is associated with a research paper.

Use Cases

• Analyze interaction dynamics between human developers and the 16 identified AI agents within code review conversations.
• Empirically assess review feedback quality across different AI agent types present in the dataset.
• Study conversation patterns and outcomes from the 54,330 pull requests to understand review effectiveness.
• Compare the nature of code review discussions originating from human versus AI agent participants.

Strengths

• Large scale with 278,790 conversation instances.
• Drawn from a substantial base of 54,330 pull requests across 300 real-world GitHub repositories.
• Captures a diverse set of 16 identified AI agents for comparative analysis.

Limitations

• Limited to 300 popular GitHub repositories, which may not represent code review practices in smaller or private projects.
• The dataset's focus on identified AI agents may miss interactions with other automated tools or unlabeled AI participation.
• Conversations are from open-source projects, potentially introducing bias towards specific programming languages, domains, or collaborative cultures.

Provenance

Source

54,330 pull requests from 300 popular open-source GitHub repositories.

Collection Method

Collection of real-world inline code review conversations.

Freshness

Last updated 2026-03-24.

Geography

Region tag indicates 'us', but specific geographic coverage is not detailed.