5. Evaluation

Overview

leADS can be evaluated using a pre-trained model (see Training). A pre-trained model ("leADS.pkl") made available to users that were trained on the Enzyme Commission (EC) number indices with embedding (biocyc21_Xe.pkl) and the pathway indices (biocyc21_y.pkl) data.

Note: As before make sure to put the source code leADS (Installing leADS) into the same directory as explained in Download files. Additionally, create a log and result (if you have not already created one during pathway prediction) folder in the same leADS_materials/ directory. The final structure should look like this:

leADS_materials/
	├── objectset/
        │       └── ...
	├── model/
        │       └── ...
	├── dataset/
        │       └── ...
	├── result/
        │       └── ...
	└── leADS/
                └── ...

For all experiments, using a terminal navigate to the src folder in the leADS directory and then run the commands. To display leADS's running options use: python main.py --help. It should be self-contained.

Input:

The essential input file used for evaluation are two matrices: [DATANAME]_X*.pkl and the [DATANAME]_y.pkl

Other files required for preprocessing:

1. bicycle.pkl
2. pathway2ec.pkl
3. pathway2ec_idx.pkl
4. pathway2vec_embeddings.npz
5. hin.pkl

Command:

python main.py --evaluate --pred-labels --soft-voting --X-name "golden_Xe.pkl" --y-name "golden_y.pkl" --dsname "golden" --file-name "leADS_ey_1" --model-name "leADS" --num-jobs 2

python main.py \
--evaluate \
--pred-labels \
--soft-voting \
--X-name "[DATANAME]_X*.pkl" \
--y-name "[DATANAME]_y.pkl" \
--file-name "[save file name]" \
--dspath "[absolute path to the dataset directory (e.g. dataset)]" \
--rspath "[absolute path to the result directory (e.g. result)]" \
--batch 50 \
--num-jobs 2