PoFBench: Policy-on-the-Fly Benchmark for AI Safety Filters

PoFBench is a test-only benchmark designed to measure performance of policy-based custom filters in LLM-powered systems. It contains harmful content for AI filter evaluation, including biased expressions, crime-related scenarios, and jailbreak attempts. The dataset was created by SamsungSDS-Research and was last updated on 2026-05-06.

Use Cases

• Evaluating the effectiveness of custom AI safety filters based on harmful content examples.
• Benchmarking policy-based filtering systems against biased expressions and jailbreak attempts.
• Testing LLM-powered systems' resilience to crime-related scenarios described in the dataset.
• Comparing performance of different safety taxonomies using the benchmark's test-only structure.

Strengths

• Benchmark focuses on policy-based custom filters, a distinct approach from fixed taxonomies.
• Dataset contains specific harmful content types: biased expressions, crime-related scenarios, and jailbreak attempts.
• Created by SamsungSDS-Research, indicating institutional backing.

Limitations

• Description metadata is limited; actual data quality requires manual inspection after download.
• Column-level documentation is absent; field semantics must be inferred after download.
• Row count is unknown, which may limit suitability assessment.

Provenance

Source

SamsungSDS-Research

Freshness

Last updated 2026-05-06 13:03:58; freshness should be verified.