You are what you're for: Essentialist categorization in large language models

This repository contains the experiments, data, analyses, and figures for the paper "You are what you're for: Essentialist categorization in large language models" by Siying Zhang, Jingyuan She, Tobias Gerstenberg and David Rose.

The preprint can be found here

Contents:

• Introduction
• Pre-registrations
• Repository structure

Introduction

How do essentialist beliefs about categories arise? We hypothesize that such beliefs are transmitted via language. We subject large language models (LLMs) to vignettes from the literature on essentialist categorization and find that they align well with people when the studies manipulated teleological information -- information about what something is for. We examine whether in a classic test of essentialist categorization -- the transformation task -- LLMs prioritize teleological properties over information about what something looks like, or is made of. Experiments 1 and 2 find that telos and what something is made of matter more than appearance. Experiment 3 manipulates all three factors and finds that what something is for matters more than what it's made of. Overall, these studies suggest that language alone may be sufficient to give rise to essentialist beliefs, and that information about what something is for matters more.

Pre-registrations

• The pre-registrations for all experiments may be accessed via the Open Science Framework here

Repository structure

├── analysis
├── code
│   ├── experiments
│   └── python
├── data
│   ├── analysis_of_prior_work
│   ├── experiment1
│   ├── experiment2
│   └── experiment3
├── figures
│   ├── analysis_of_prior_work
│   ├── experiment1
│   ├── experiment2
│   └── experiment3
└── writeup

• analysis/ contains all the code for analyzing data and generating figures, written in R. (view a rendered html file here).
• code/ contains all the materials and code for the experiments.
  • experiments contains materials for each experiment that was run.
    • analysis_of_prior_work was run on GPT-3 (Model: text-curie-001) and BLOOM. exp1_tasks_collection_essentialism.csv was read into the python script and the language models' responses were appended to the end of the CSV file. The completed file was saved to data/. Likewise for the experiment1, experiment2 and experiment3. ** Note that the data for analysis_of_prior_work were saved by studies.
    • experiment1, experiment2 and experiment3, were run on GPT-3 (Model: text-davinci-002) and BLOOM.
  • python contains scripts that were used to run the experiments.
    • aopw_gpt3_response_generator.ipynb, experiment1_gpt3_response_generator.ipynb, experiment2_gpt3_response_generator.ipynb, experiment3_gpt3_response_generator.ipynb, aopw_bloom_response_generator.ipynb, experiment1_bloom_response_generator.ipynb, experiment2_bloom_response_generator.ipynb and experiment3_bloom_response_generator.ipynb are for generating language models' responses.
    • aopw_gpt3_answer_retrieval_cached.ipynb and aopw_bloom_answer_retrieval_cached.ipynb are for the data processing task in analysis_of_prior_work by training GPT-3 (Model: text-curie-001) to retrieve single-word responses from the full text responses that were generated by querying the language models. NOTE: These two files were ONLY used for processing the data in analysis_of_prior_work.
    • aopw_alignment_analysis_of_prior_work_GPT3.ipynb and aopw_alignment_analysis_of_prior_work_BLOOM.ipynb are for obtaining the degree of alignment with people's categorization judgments by studies in analysis_of_prior_work.
• data/ contains all data from all experiments.
  • For analysis_of_prior_work:
    • aopw_gpt3_response_by_study and aopw_bloom_response_by_study hold the unprocessed raw data that resulted from querying the language models.
    • aopw_gpt3_response_by_study_answer_retrieved and aopw_bloom_response_by_study_answer_retrieved hold the processed data that were obtained after the answer retrieval task, which were used for generating the figures in figures/.
  • For experiment1:
    • experiment1_gpt3_unprocessed and experiment1_bloom_unprocessed hold the unprocessed data that resulted from querying the language models.
    • experiment1_gpt3_answer_retrieved and experiment1_bloom_answer_retrieved hold the processed data that were obtained after the answer retrieval task. NOTE: For this and all subsequent experiments, two independent coders extracted item names from responses, cross-checking each other's work to ensure accuracy. Any discrepancies were marked as "unsure."
  • Likewise for experiment2 and experiment3.
• figures/ contains all the figures from the project (generated using the script in analysis/).