Chatbot Learning Assistant Study: Student Perceptions and Performance Over Five Years

A five-year, multi-cohort study followed 240 first-year university students to evaluate the impact of a mobile chatbot learning assistant. Pavel Smutny authored this research document, which compares a decision-tree-based chatbot with an LLM-driven version on learning outcomes, student perceptions, and motivation. The dataset, last updated in May 2026, contains the full study results and discussion in a 1.5 MB document.

Use Cases

• Analyze student perceptions of chatbot usability and accessibility based on the study's qualitative findings.
• Compare the effects of rule-based versus LLM-driven chatbot designs on student engagement based on the multi-cohort results.
• Study the relationship between chatbot support and academic performance in a first-year university course based on the quasi-experimental design.

Strengths

• Study design includes 240 students over five course iterations, providing longitudinal data.
• Explicitly compares two distinct chatbot technologies: a decision-tree-based system and an LLM-driven version.
• Results include quantitative performance data and qualitative student perception evaluations.

Limitations

• The core data is a 1.5 MB DOCX file; row count and column-level data structure are unknown.
• Description metadata is limited; actual data quality and granularity require manual inspection after download.
• Data may reflect temporal and institutional bias inherent to the single-course, five-year study design.

Provenance

Source

figshare

Collection Method

Quasi-experimental, multi-cohort study with natural assignment of students to control and experimental groups.

Time Range

Study conducted over a five-year period (specific years not stated).

Freshness

Last updated 2026-05-21 15:06:42; freshness should be verified.