Synthetic image-text pairs across harmful and safe categories for text-to-image safety alignment. This dataset enables Direct Preference Optimization (DPO) to train safety experts that guide generative processes away from specific safety-related concepts as introduced in the AlignGuard paper.
Use Cases
- Train a text-to-image model using Direct Preference Optimization (DPO) with the provided harmful and safe image-text pairs.
- Develop 'safety expert' adapters to steer generative outputs away from the safety-related concepts defined in the dataset.
- Evaluate model safety by comparing generated images against the harmful and safe prompt pairs.
Strengths
- Includes paired image-text data categorized into harmful and safe classes.
- Specifically formatted for Direct Preference Optimization (DPO) in text-to-image (T2I) models.
- Designed for integration with Stable Diffusion XL (SDXL) architectures for safety fine-tuning.