Delivering a discrete image encoder and decoder trained on 100,000 scientific figure images from the Paper2Fig100k collection. It utilizes a specialized OCR Perceptual loss to optimize the reconstruction and generation of legible text within complex graphical figures.
Use Cases
- Generate discrete image tokens for scientific figures to be used in transformer-based generative models
- Reconstruct figure images from latent codes using the provided detokenizer weights
- Improve text legibility in image synthesis tasks by applying the OCR Perceptual loss function
Strengths
- Trained on 100,000 scientific figures from the Paper2Fig100k dataset
- Includes a discrete image tokenizer and detokenizer based on the VQGAN architecture
- Implements a specialized OCR Perceptual loss for text-within-image generation
- Forked from the CompVis/taming-transformers implementation