Tfrecord: Standalone TFRecord Reader and Writer for PyTorch

Developed by vahidk and updated in May 2025, this repository provides a standalone TFRecord reader and writer for PyTorch. It enables the consumption of TensorFlow-native data formats within PyTorch pipelines without requiring a full TensorFlow installation.

Use Cases

• Reading sharded data using the TFRecordDataset class for efficient PyTorch training
• Generating index files for fast random access using the tfrecord2idx utility
• Defining data schemas using the description parameter to map TFRecord features to PyTorch tensors

Strengths

• MIT License allows for commercial and private use
• Zero TensorFlow dependency for reading TFRecord files
• Direct compatibility with PyTorch DataLoader

Limitations

• Does not contain any pre-packaged data records or labels
• Requires manual generation of index files for random access functionality

Provenance

Source

GitHub repository by vahidk

Freshness

Last updated May 2025.