The official implementation of the method discussed in the paper Improving Spoken Language Identification with Map-Mix(accepted at ICASSP 2023).
Abstract: The pre-trained multi-lingual XLSR model generalizes well for language identification after fine-tuning on unseen languages. However, the performance significantly degrades when the languages are not very distinct from each other, for example, in the case of dialects. Low resource dialect classification remains a challenging problem to solve. We present a new data augmentation method that leverages model training dynamics of individual data points to improve sampling for latent mixup. The method works well in low-resource settings where generalization is paramount. Our datamaps-based mixup technique, which we call Map-Mix improves weighted F1 scores by 2% compared to the random mixup baseline and results in a significantly well-calibrated model.
Each baseline methods mentioned in the paper(ie: pretrained-baselines, mixup-baselines, datamap-mixup, map-mix) are added to different branches of this repository.
- Pretrained Baselines(wav2vec2, Hubert, XLSR)
- Mixup Baselines(static, latent-random, latent-within, latent-across)
- Generation of Datamaps
- Datamap Mixup (easy mixup, hard mixup, amb+easy mixup)
- Map-Mix
Download the LRE-2017 Dataset and update the dataset path in config.json file.
git clone https://github.com/skit-ai/Map-Mix.git
cd Map-Mix
# checkout to the required method's branch
git checkout baselines
pip install -r requirements.txt
The training and testing script for all the baselines and methods are same. Edit the config.json file for the respective branch with the training hyperparameters and dataset path.
# Train the model
python train.py
# Evaluate the model and generate test prediction csv files
python test.py
# Get the evaluation metrics
# add the csv path to the test_metrics.py file
python test_metrics.py
Nil
@misc{https://doi.org/10.48550/arxiv.2302.08229,
doi = {10.48550/ARXIV.2302.08229},
url = {https://arxiv.org/abs/2302.08229},
author = {Rajaa, Shangeth and Anandan, Kriti and Dalmia, Swaraj and Gupta, Tarun and Chng, Eng Siong},
keywords = {Machine Learning (cs.LG), Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
title = {Improving Spoken Language Identification with Map-Mix},
publisher = {arXiv},
year = {2023},
copyright = {arXiv.org perpetual, non-exclusive license}
}