A Neural Local Coherence Model for Text Quality Assessment

A neural local coherence model based on semantic changes across sentences in a text. Your interest to this project is very appreciated. Please cite this paper if you use the above code. Also don't forget to give it a Github star (on top right).

Setup

• OS: linux2
• GCC 7.2.0
• Python 2.7.14
• PyTorch 0.1.12_2
• More info: spec-file.txt

Procedure

• Prepare data
• Run main_making.py

Data

For the essay scoring experiments, we use the ASAP dataset to evaluate our system. This dataset (training_set_rel3.tsv) can be downloaded from here. After downloading the file, put it in the data directory and create training, development and test data using preprocess_asap.py script:

cd data
python preprocess_asap.py -i training_set_rel3.tsv

Publication

Mohsen Mesgar and Michael Strube. 2018. A Neural Local Coherence Model for Text Quality Assessment. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing.

FAQ

(1) How can one create the *.pkl files required by main_masking.py for the essay scoring task?

• These features were generated by the EASE system and are used only for the essay scoring task. This code (https://github.com/edx/ease/blob/master/ease/feature_extractor.py) helps to generate the files.