Artha Benchmark: Personal-Finance Reasoning for LLM Agents

The Artha Personal-Finance Reasoning Benchmark is a reproducible, model-agnostic benchmark for evaluating LLM-based personal-finance agents. It assesses answer quality and grounding/hallucination resistance over a user's full transaction ledger, using a four-dimension rubric scored by a three-model-family LLM judge panel. The dataset was created by Tej-Katika and last updated on Hugging Face on 2026-07-02.

Use Cases

• Benchmarking LLM reasoning on multi-hop financial queries based on transaction ledger data mentioned in the description
• Evaluating model resistance to false premises and hallucinations in a financial context as described
• Training or fine-tuning financial agents on aggregative reasoning tasks over banking and investment data

Strengths

• Benchmark is designed to be reproducible and model-agnostic, as stated in the description
• Evaluation uses a four-dimension rubric and a three-model-family LLM judge panel
• Focuses on two axes not jointly exercised by existing financial-QA benchmarks: answer quality and grounding/hallucination resistance

Limitations

• Column-level documentation is absent; field semantics must be inferred after download
• Row count, file formats, and sample data are unknown, which may limit suitability assessment
• Description metadata is limited; actual data quality requires manual inspection after download

Provenance

Source

Tej-Katika on Hugging Face

Collection Method

Likely constructed for benchmarking purposes; specific gathering method is not detailed.

Freshness

Last updated 2026-07-02 05:55:35; freshness should be verified