Im2Latex

Neural Model converting Image to Latex.

As the picture shows, given an input image, a CNN and RNN encoder is applied to extract visual features firstly. And then the encoded features are used by an RNN decoder with attention mechanism to produce final formulas.

Install Prerequsites

pip3 install -r requirements.txt

Quick Start

We provide a small preprocessed dataset (./sample_data/) to check the pipeline. To train and evaluate model in this sample dataset:

python3 train.py

Train on full dataset

Download the prebuilt dataset from Harvard and use their preprocessing scripts found here.

After that, to train and evaluate model on full dataset, you can just pass the full dataset path to train.py:

python3 train.py --data_path=FULL_DATASET_PATH

Reference

What You Get Is What You See:A Visual Markup Decompiler

The Official Torch implemention

A Tensorflow implemention