This repository has code to:
- Approximate the memory of neural networks with a mathematical function.
- Analyze properties of words that affect memory.
MathematicalModelling.ipynb - This notebook approximates the memory of a neural network with a mathematical funtion.
SyntheticDataAnalysis.ipynb - This notebook trains bert using a synthetic dataset.
extract_bert_features.ipynb - create train, validation, and test dataloader with bert representations.
extract_lstm_features.ipynb - create train, test, validation dataloaders with lstm representations.
fast_lstm.ipynb - use lstm language model using fastai.
lm_data_generation.ipynb - data generation using wikitext.
mem_with_bert_train.ipynb - train bert with the natural language dataset created from wikitext.
mem_with_lstm_train.ipynb - train lstm with the natural language dataset created from wikitext.
memory_analysis.ipynb - analyze the factors on which memory depends.
synthetic_data_using_lstm_nonoverlap_query.ipynb - train lstm with synthetic data.
synthetic_data_using_transformer.ipynb - train transformer with syntheic data.
synthetic_dataset.ipynb - creation of synthetic dataset and training a cnn and lstm on it.
synthetic_dataset_generation.ipynb - generation of synthetic dataset.