MTEB-PT Results: Brazilian-Portuguese Text Embedding Benchmark Scores

96 models were evaluated on 27 native Brazilian-Portuguese tasks in the MTEB(por, v2) benchmark. The dataset stores canonical results for approximately 75 open-weight and 20 closed/API models, plus a random baseline, as generated by the mteb library on pinned-SHA datasets. It was created by mteb-pt and last updated on 2026-06-29.

Use Cases

• Compare model performance across multiple tasks based on the 27 native Portuguese evaluation datasets.
• Benchmark new text embedding models against existing results based on the 96-model evaluation suite.
• Analyze the effectiveness of open-weight versus closed/API models for Portuguese NLP based on the described model categories.
• Identify suitable embedding models for specific Portuguese language tasks based on the structured task-by-task results.

Strengths

• Evaluates 96 distinct models, providing a broad comparison landscape.
• Covers 27 native Brazilian-Portuguese tasks, avoiding potential issues with machine-translated benchmarks.
• Results are generated using a standardized evaluation library (mteb) on pinned dataset versions for reproducibility.

Limitations

• Column-level documentation is absent; field semantics must be inferred after download.
• Row count is unknown, which may limit suitability assessment.
• Description metadata is limited; actual data quality and structure require manual inspection after download.

Provenance

Source

huggingface

Collection Method

Results generated by the mteb evaluation library on pinned-SHA datasets.

Freshness

Last updated 2026-06-29 11:41:32; freshness should be verified.

Geography

Brazilian-Portuguese language focus.