Ml Pyxis: Tensor Data Management with LMDB and Msgpack

Managing an unspecified number of tensor records for machine learning, this framework by vicolab provides high-speed data access via LMDB storage. Released in 2020, it facilitates the management of multi-dimensional data for deep learning applications.

Use Cases

• Accelerated data loading for training deep learning models using **tensors** as input features
• Fast retrieval of specific data samples using **LMDB keys** for indexing
• Efficient serialization of high-dimensional **tensors** using Msgpack for storage

Strengths

• MIT license
• LMDB-based fast reading speeds
• Msgpack serialization support

Limitations

• Does not contain a predefined set of data records
• Temporal staleness with the last update in 2020
• Requires specific Python dependencies for LMDB access

Provenance

Source

vicolab GitHub repository

Freshness

Last updated in September 2020.