Long-Term Memory Embedding Benchmark for Retrieval Models

LMEB is a standardized benchmark for evaluating the long-term memory retrieval capabilities of text embedding models. Created by KaLM-Embedding, it provides reproducible tests to identify models suitable for memory-augmented systems like OpenClaw. The dataset was last updated in March 2026.

Use Cases

• Rank embedding models based on performance on long-context retrieval tasks defined by the benchmark.
• Evaluate model robustness for memory-augmented systems by testing on LMEB's standardized memory retrieval challenges.
• Compare model accuracy on text retrieval over extended contexts using the benchmark's evaluation suite.

Strengths

• Benchmark designed for reproducible evaluation of a specific model capability.
• Focuses on a defined niche: long-term memory retrieval for embedding models.

Limitations

• Specific dataset size, row count, and internal structure are not publicly documented.
• The benchmark's scope is narrow, focusing only on memory retrieval rather than general embedding performance.

Provenance

Source

KaLM-Embedding on Hugging Face.

Collection Method

Constructed as an evaluation benchmark; specific data gathering method is null.

Freshness

Last updated March 2026.