FineVision-vlmbench-mini: 128 Rows for VLM Inference Performance Benchmarking

FineVision-vlmbench-mini is a 128-row subset of the FineVision dataset, designed for benchmarking Vision-Language Model inference performance. The dataset was created by the author vlm-run and last updated on 2026-02-08. It is intended to provide a realistic and diverse workload for measuring metrics like tokens per second and VRAM usage.

Use Cases

• Benchmarking VLM inference latency based on the dataset's stated purpose for measuring time-to-first-token.
• Measuring VLM throughput under batching based on the description's focus on performance under varied workloads.
• Profiling VRAM usage during VLM inference based on the dataset's design for evaluating system resource consumption.
• Comparing inference performance across different VLM architectures using a standardized, small-scale multimodal workload.

Strengths

• Contains 128 randomly selected rows, providing a small-scale, manageable sample for performance testing.
• Designed to span four input modalities, offering a diverse workload for benchmarking.
• Explicitly created for a specific, measurable purpose: benchmarking VLM inference performance metrics.

Limitations

• Row count is unknown for the full source dataset, which may limit suitability assessment for larger-scale tasks.
• Column-level documentation is absent; field semantics must be inferred after download.
• Description metadata is limited; actual data quality and content require manual inspection after download.

Provenance

Source

Subset of HuggingFaceM4/FineVision dataset, hosted on Hugging Face.

Collection Method

128 rows were randomly selected from the source dataset.

Freshness

Last updated 2026-02-08 19:08:27; freshness should be verified.