Work We Use

• LLMD (Fryer, 2022), to augment data using large language models;

• INLP (Ravfogel, 2020) and RLACE (Ravfogel, 2022), to find key directions in neural activations;

• Stereoset (Nadeem, 2020), a large collection of stereotypes;

• The “Double bind experiment” (Heilman, 2007), an experiment about bias in humans which can also be conducted with large language models, and (May, 2019), which provides the exact data we use;

• OpenAI’s API, to run inferences on large languages models;

• De Gibert, 2018, which provides data about hate speech;

• nltk, and Jörg Michael’s gender.c, which contain datasets about the gender and origin of first names;

• BigBench’s Social Bias from Sentence Probability, which provides evaluation data and metrics.