Examples of Vignettes Created in Study 1 for LLM Risk Assessment

A collection of clinical vignettes constructed from participants' lived experience for evaluating large language models in psychosocial risk assessment. The dataset, authored by Laura M. Vowels and shared on figshare, contains examples used to benchmark GPT-4 and Claude 3 Sonnet on detecting suicide risk, intimate partner violence, and substance misuse. It was last updated on April 27, 2026.

Use Cases

• Benchmarking LLM performance in detecting psychosocial risk domains based on the described vignettes.
• Analyzing model agreement with human-rated severity for suicide, IPV, and substance misuse.
• Training or fine-tuning supervised risk assessment systems based on structured vignette examples.

Strengths

• Vignettes are constructed from participants' lived experience, providing realistic clinical scenarios.
• Dataset is openly licensed under CC-BY-4.0, permitting reuse and redistribution.
• File format is XLS, which is widely accessible for inspection and analysis.

Limitations

• Row count and column-level documentation are unknown, limiting suitability assessment.
• The dataset is very small at 13.5 KB, indicating a limited scope of examples.
• Data may reflect the specific bias inherent to the vignette construction method described in the study.

Provenance

Source

Laura M. Vowels via figshare.

Collection Method

Vignettes were constructed from participants' lived experience as part of a research study.

Time Range

The study timeframe is not specified in the provided metadata.

Freshness

Last updated 2026-04-27 17:45:15; freshness should be verified.

Geography

Geographic coverage is not specified in the provided metadata.