Synthetic Multi-Turn Customer Service QA for Small Language Model Evaluation

A synthetic dataset constructed to evaluate nine instruction-tuned Small Language Models (SLMs) against three commercial Large Language Models (LLMs) for context-summarized multi-turn customer-service question answering. The dataset was created by Lakshan Cooray and published on figshare in June 2026 under a CC-BY-4.0 license. It includes a structured evaluation framework combining quantitative metrics with human and LLM-as-a-judge assessments.

Use Cases

• Benchmarking Small Language Model performance on multi-turn dialogue tasks based on the described synthetic QA data.
• Training or fine-tuning models for context-summarized customer service QA using the described parameter-efficient fine-tuning approach.
• Studying dialogue continuity and contextual alignment in customer interactions based on the conversation stage-based qualitative analysis mentioned.
• Evaluating the trade-offs between model size and performance in resource-constrained environments as described in the comparative study.

Strengths

• Includes a structured evaluation framework with both quantitative metrics and qualitative assessments (human and LLM-as-a-judge).
• Explicitly compares nine instruction-tuned SLMs against three commercial LLMs, providing a direct performance benchmark.
• Dataset is openly available under a permissive CC-BY-4.0 license.

Limitations

• Dataset size is 390.4 KB, indicating a very limited scope and likely small number of synthetic dialogue examples.
• Row count and column-level documentation are unknown, making suitability assessment difficult.
• Data is synthetic, which may not fully capture the complexity and distribution of real-world customer service conversations.

Provenance

Source

Lakshan Cooray via figshare.

Collection Method

Generated via a synthetic data construction pipeline described in the associated research.

Freshness

Last updated 2026-06-02 05:40:21.