Improving Domain Specific QA: Inter IIT Tech Meet 11.0, 2023 Retriever Approach Top-1 Accuracy Top-5 Accuracy Cosine Similarity b/w MPNet (multi-qa-mpnet-base) Embeddings 63.57 88.4 BM25 Scores 58.57 79.82 MPNet Embeddings + BM25 70.67 89.63 Accuracies on SQuAD-V2 dev set with theme information Reader Architecture F1 EM BERT-base 74.67 71.15 ELECTRA-base 81.71 77.60 DeBERTa-V3-base 87.41 83.92 F1 and EM on SQuAD-V2 dev set Domain Adaptation Retriever Approach Top-1 Accuracy Top-5 Accuracy multi-qa-mpnet-base 63.57 88.4 GPL (multi-qa-mpnet-base) 66.5 86.4 LaPraDoR (checkpoint not trained on SQuADV2 Retrieval) 51.2 79.9 Reader Approach F1 EM BERT-base zero shot 74.67 71.15 CAQA (Synthetic - QAGen-T5-base) 72.42 68.91 CAQA (No Synthetic Data) 76.27 72.87 QADA (4 epochs) 76.50 73.23 Approach F1 EM DeBERTa-V3-base zero shot 87.41 83.92 CAQA (Synthetic - QAGen-T5-base) 86.12 82.68 CAQA (No Synthetic Data) 88.93 85.07