LLM-JP Instructions Jculture: Japan-Specific Question-Answer Pairs for Evaluation

llm-jp provides a Japan-specific question-answer evaluation dataset created by native Japanese annotators. The dataset collects questions where the correct answer differs between Japan and other countries, such as rules for which side of the road to drive on. It was last updated on 2026-04-17.

Use Cases

• Evaluate LLM performance on Japan-specific cultural knowledge based on the question-answer pairs.
• Fine-tune language models for improved accuracy in Japanese cultural contexts based on the annotated examples.
• Benchmark cross-cultural reasoning capabilities of AI systems based on the defined 'Japan-specific' criteria.
• Study model bias or knowledge gaps regarding regional information based on the comparative question structure.

Strengths

• Created by native Japanese annotators, ensuring cultural and linguistic authenticity.
• Focuses on questions where answers differ between Japan and other countries, providing a clear evaluation criterion.
• Last updated date of 2026-04-17 is provided.

Limitations

• Row count, column structure, and file formats are unknown, which limits suitability assessment.
• Column-level documentation is absent; field semantics must be inferred after download.
• License information is unknown, which may restrict usage.

Provenance

Source

llm-jp on Hugging Face.

Collection Method

Collected by native Japanese annotators.

Time Range

null

Freshness

Last updated 2026-04-17 04:00:37; freshness should be verified.

Geography

Japan