Bias Decomposition in LLMs — Repository

This repository accompanies our research paper on bias detection and decomposition in large language models (LLMs) entitled "Unpacking Bias in LLMs: A Causal Testing Perspective", which presents CFTA (Causal Fairness Testing and Analysis).

The paper is submitted for publication to the ACM Transactions on Software Engineering and Methodology.

This repository provides code, synthetic datasets, prompts, and experimental results that support the study's findings. The repository is designed to make our experiments reproducible and extendable for further research on fairness testing.

CFTA measures how models encode protected attributes (gender, race) into sentiment / toxicity scores over synthetic data generated under a structural causal model (SCM), then decomposes the effect into Direct / Indirect / Spurious components. Sentences are produced by three interchangeable text generators — two LLM generators driven by prompts (CHATGPT4, GEMINIPRO) and a deterministic TEMPLATE generator used as a control — and scored by ten sentiment / toxicity scorers (five local Hugging Face models and five API-hosted models).

Repository Overview

This repository contains the code, synthetic datasets, and experimental results supporting our study on fairness testing in LLMs. It is organized as follows:

CODE/: Python and R scripts.

• common/: shared helpers (path resolver and SCM definitions, the resumable scoring engine, dataset builders) used across the whole pipeline.
• run_sa.py: unified sentiment / toxicity scorer (RQ2).
• run_experiments.py: unified launcher for the additional experiments (E1, E3, gc).
• GENDER/: SCM with gender as the protected attribute. Includes code for dataset generation and analyses for RQ1–RQ3.
• RACE/: SCM with race as the protected attribute. Includes code for dataset generation and analyses for RQ1–RQ3.
• EXPERIMENTS/: the additional experiments (template generator, DAG sensitivity, generator comparison).

DATASET/: Datasets generated by CFTA, organized by protected attribute and by generator. The synthetic structured data is shared across generators (it lives under DATASET/<ATTR>/SYNTHETIC_DATA/), while each generator keeps its own sentences and merged corpus.

• GENDER/
  • SYNTHETIC_DATA/: Synthetic structured datasets split by scenario, from which sentences are then generated (see the paper).
  • CHATGPT4/, GEMINIPRO/, TEMPLATE/, each with:
    • GENERATED_SENTENCES/: Generated sentences split by scenario.
    • COMPLETE_DATASET/: Full dataset (one aggregated file, all scenarios, with a Scenario column). The ChatGPT4 folder also includes all_scenarios_MG-Table.pdf (gender) / all_scenarios_MR-Table.pdf (race), which document coefficients, interpretations, and expected bias for the scenarios.
• RACE/: same structure.

PROMPT/: Prompts used to generate the LLM-based datasets (prompt_gender.txt, prompt_race.txt).

RESULT/: Experimental results, organized by attribute and generator.

• GENDER/, RACE/
  • <GENERATOR>/: outputs for RQ1–RQ3, plus the DAG-sensitivity experiment (E1_dag_sensitivity/).
  • GENERATOR_COMPARISON/: the generator-comparison experiment (attribute-level).

Throughout, <ATTR> is GENDER or RACE and <GENERATOR> is CHATGPT4, GEMINIPRO, or TEMPLATE. There are 14 scenarios for gender and 12 for race.

Run every command from the repository root. All paths are resolved relative to the repository root automatically, so the scripts work unchanged from a freshly moved copy of the repository.

Almost every script accepts the same two switches, --attr and --generator, which can also be provided through the ATTR / GENERATOR environment variables. The --generator switch defaults to CHATGPT4.

How to Run (Step-by-Step Execution)

1. Set up the environment

Use Python 3.11.0 and install all required dependencies:

pip install -r requirements.txt

This covers all Python steps (pandas, transformers / torch for the local sentiment models, openai / requests for the hosted models, matplotlib + scipy for the analyses).

The causal decomposition (RQ2/RQ3 and experiments E1 / gc) is written in R 4.6.1. From an R console install the required packages, including faircause (Plečko), which is installed from GitHub:

install.packages(c("readr","fs","dplyr","tidyr","purrr","tibble",
                   "stringr","ggplot2","progress"))
install.packages("remotes")
remotes::install_github("dplecko/CFA")   # provides the faircause package

The run_experiments.py launcher discovers Rscript automatically (from the PATH, from the RSCRIPT environment variable, or from the standard C:\Program Files\R\... location). If you launch an R script directly and Rscript is not on your PATH (common on Windows / VS Code), add it for the current session:

$env:Path += ";C:\Program Files\R\R-4.6.1\bin"
Rscript --version

2. Generate synthetic datasets

To create the synthetic datasets for the GENDER SCM, use the script in CODE/GENDER/SYNTHETIC_DATA/.

What the script does

• Samples 800 individuals and builds: Gender (protected attribute), Age, Education, Employment, and Context (the human-readable encoding of the latent variable U).
• Writes the output to DATASET/GENDER/SYNTHETIC_DATA/synthetic_data_s{i}.csv.

Scenarios

• There are 14 scenarios in total for gender (12 for race).
• To reproduce exactly the same results of the paper: for each scenario, set the causal-model coefficients (those controlling the influence of the latent variable and the protected attribute) to the same values as in the paper, passing them on the command line, so that each scenario produces the corresponding CSV in DATASET/GENDER/SYNTHETIC_DATA/ (e.g. synthetic_data_s2.csv, …, synthetic_data_s14.csv).
• To replicate in a different context (hence generate data according to different distributions), change the causal coefficients as needed and run the script.

Run commands

# GENDER SCM — scenario s1 with the paper's coefficients (example)
python CODE/GENDER/SYNTHETIC_DATA/synthetic_data_gender.py --scenario s1 \
  --lambda-gender 0.0 --mu-age 0.0 --alpha1 0.0 --alpha2 0.0
# then change --scenario and the coefficients and rerun for s2, s3, …, s14

# RACE SCM (analogous)
python CODE/RACE/SYNTHETIC_DATA/synthetic_data_race.py --scenario s1 \
  --lambda-race 0.0 --mu-age 0.0 --alpha 0.0 --beta 0.0
# then rerun for s2, s3, …, s12

The RACE records contain Race, Age, Prior Convictions, and Context. Outputs are saved under DATASET/RACE/SYNTHETIC_DATA/.

3. Generate sentences from the structured records

LLM generators (CHATGPT4, GEMINIPRO). Copy the prompt as-is from the PROMPT/ folder into your LLM. Provide the LLM with both:

• the prompt file (PROMPT/prompt_gender.txt or PROMPT/prompt_race.txt), and
• the synthetic dataset CSV generated in Step 2 for the selected scenario.

In the prompt, only change the scenario identifier in the file-name instruction, e.g.:

"For each record, generate a single sentence following the rules and style of the examples. Then, save everything in a file called generated_sentences_s{i}.csv, which must contain the following columns: Gender, Age, Education, Employment, and Sentence."

Replace {i} with the scenario number (e.g. s1, s4, s12, …). Save the result to DATASET/<ATTR>/<GENERATOR>/GENERATED_SENTENCES/generated_sentences_s{i}.csv, with columns Gender, Age, Education, Employment, Sentence (gender) or Race, Age, Prior Convictions, Sentence (race).

TEMPLATE generator (deterministic control). Instead of calling an LLM, this generator fills a fixed template over the same structured records, producing a corpus that the downstream pipeline consumes with no changes — only the generator name differs. See Step 8.

Merge into COMPLETE_DATASET. For convenience, merged files (all scenarios together, with a Scenario column) can be built for any generator:

python CODE/common/build_complete_dataset.py --generator GEMINIPRO --attr all

This writes DATASET/<ATTR>/<GENERATOR>/COMPLETE_DATASET/generated_sentences_<attr>_complete.csv.

4. RQ1 — Quality & Consistency checks

This step evaluates the quality of the generated sentences and their consistency with the original synthetic records, for each scenario and in aggregate.

Input (per scenario)

• DATASET/<ATTR>/SYNTHETIC_DATA/synthetic_data_s{i}.csv
• DATASET/<ATTR>/<GENERATOR>/GENERATED_SENTENCES/generated_sentences_s{i}.csv

What the script does

• Quality metrics on generated sentences: average word length per sentence, unique token counts (bootstrap samples), and readability scores (Flesch–Kincaid, ARI, Gunning–Fog).
• Consistency checks (row by row): whether each sentence contains the right fields (Gender, Age, Education, Employment), and whether the hidden Context is mentioned.
• Aggregate analysis (across all available scenarios): raw vs deduplicated corpora, aggregate quality metrics (CSV) and a metrics-summary plot (PNG), rarefaction curves (CSV + PNG), and the duplicate rate.

Where results are saved

• Per-scenario: RESULT/<ATTR>/<GENERATOR>/RQ1/scenario_s{i}/quality_metrics_s{i}.csv and consistency_report_s{i}.csv.
• Aggregate: RESULT/<ATTR>/<GENERATOR>/RQ1/aggregate/ (corpus CSVs, duplicate_rate.txt, metrics_summary.png, rarefaction_curve.png, …).

Run commands

python CODE/GENDER/RQ1/quality_and_consistency_sweep.py --generator GEMINIPRO
python CODE/RACE/RQ1/quality_and_consistency_sweep.py   --generator GEMINIPRO

5. RQ2 — Sentiment / Toxicity Analysis (SA)

Sentiment / toxicity analysis on the generated sentences is performed by ten scorers.

Models

• BERT-like (Hugging Face, local): cardiffnlp_twitter_roberta_base_sentiment_latest, distilbert_base_uncased_finetuned_sst2_english, siebert_sentiment_roberta_large_english, textattack_bert_base_uncased_SST_2, unitary_toxic_bert. The last one performs toxicity analysis (not sentiment).
• Generative LLMs (API): gpt_3_5_turbo, allam_2_7b, gemma2_9b_it, llama_3_1_8b_instant, llama3_70b.

Each model can be run through its own script under CODE/<ATTR>/RQ2/SA/, which selects the attribute automatically and accepts --generator (default CHATGPT4) plus any of the options below. Scripts iterate over the generator's generated_sentences_s{i}.csv files and write results to RESULT/<ATTR>/<GENERATOR>/RQ2/SA/<model_name>/, adding a sentiment_score / sentiment_score_positive / toxicity_score column.

Run commands (from the repository root)

# GENDER — BERT-like (Hugging Face)
python CODE/GENDER/RQ2/SA/cardiffnlp_twitter_roberta_base_sentiment_latest.py --generator GEMINIPRO
python CODE/GENDER/RQ2/SA/distilbert_base_uncased_finetuned_sst2_english.py   --generator GEMINIPRO
python CODE/GENDER/RQ2/SA/siebert_sentiment_roberta_large_english.py          --generator GEMINIPRO
python CODE/GENDER/RQ2/SA/textattack_bert_base_uncased_SST_2.py               --generator GEMINIPRO
python CODE/GENDER/RQ2/SA/unitary_toxic_bert.py                               --generator GEMINIPRO

# GENDER — Generative (APIs)
python CODE/GENDER/RQ2/SA/gpt_3_5_turbo.py          --generator GEMINIPRO
python CODE/GENDER/RQ2/SA/allam_2_7b.py             --generator GEMINIPRO
python CODE/GENDER/RQ2/SA/gemma2_9b_it.py           --generator GEMINIPRO
python CODE/GENDER/RQ2/SA/llama_3_1_8b_instant.py   --generator GEMINIPRO
python CODE/GENDER/RQ2/SA/llama3_70b.py             --generator GEMINIPRO

# RACE — use the identical scripts under CODE/RACE/RQ2/SA/

Alternatively, a single launcher scores any generator's sentences with any model or group of models:

python CODE/run_sa.py --list                                          # show all models
python CODE/run_sa.py --attr GENDER --generator GEMINIPRO --model cardiff
python CODE/run_sa.py --attr RACE   --generator GEMINIPRO --model all --resume
python CODE/run_sa.py --attr GENDER --generator GEMINIPRO --model hf --scenarios 1-3 --limit 20

Scoring is resumable: progress is checkpointed and only rows that are still missing are scored. If an API run is interrupted or rate-limited, re-run the exact same command with --resume (on by default) and it picks up where it stopped; use --force to recompute from scratch.

API keys. Set only the provider you are about to use.

# Linux / macOS
export OPENAI_API_KEY="..."   # gpt_3_5_turbo (official OpenAI endpoint)
export GROQ_API_KEY="..."     # allam_2_7b, gemma2_9b_it, llama_3_1_8b_instant, llama3_70b (Groq endpoint)
export HF_TOKEN="..."         # only if a Hugging Face model gates the download

# Windows PowerShell
$env:OPENAI_API_KEY="..."
$env:GROQ_API_KEY="..."
$env:HF_TOKEN="..."

6. RQ2 — Bias Decomposition (R)

The decomposition scripts run faircause (Plečko) over the per-scenario outputs created in Step 5 and decompose the Total Variation into Direct / Indirect / Spurious effects. They read the scored files under RESULT/<ATTR>/<GENERATOR>/RQ2/SA/<model_name>/ and select the outcome column automatically among sentiment_score, sentiment_score_positive, sentiment_score_positivo, and toxicity_score.

• Per scenario: one PDF per model × scenario in RESULT/<ATTR>/<GENERATOR>/RQ2/RSTUDIO/<model_name>/.
• Pooled over all scenarios (recommended): one PDF per model in RESULT/<ATTR>/<GENERATOR>/RQ2/RSTUDIO_ALL_SCENARIOS/<model_name>/.

Run commands

# per scenario
Rscript CODE/GENDER/RQ2/RSTUDIO/decomposition.R --generator GEMINIPRO
Rscript CODE/RACE/RQ2/RSTUDIO/decomposition.R   --generator GEMINIPRO

# pooled over all scenarios
Rscript CODE/GENDER/RQ2/RSTUDIO/decomposition_all_scenarios.R --generator GEMINIPRO
Rscript CODE/RACE/RQ2/RSTUDIO/decomposition_all_scenarios.R   --generator GEMINIPRO

unitary_toxic_bert is a toxicity model, not a sentiment scorer, so it is excluded from the causal decomposition (and from RQ3 and the experiments below). It remains available in Step 5 for the raw toxicity measurement.

7. RQ3 — Sampling-budget Sensitivity (R)

This step performs a bias-decomposition sensitivity analysis by varying the per-scenario sampling budget (e.g. 100–800 rows). For each model and scenario it reads the CSVs from Step 5, builds strata (protected attribute × covariates × age bins), runs fairness_cookbook (Plečko) on a stratified subsample, extracts Total / Direct / Indirect / Spurious, aggregates the estimates per budget, and produces a comparison plot across models.

Produced outputs go to RESULT/<ATTR>/<GENERATOR>/RQ3/, including per-model *_perSCENARIO_stratified.csv and *_AGG_curves.csv, a combined summary CSV, and a PLOTS/ figure. A model whose output folder already exists and is non-empty is skipped (no recomputation).

Run commands

Rscript CODE/GENDER/RQ3/sensitivity_gender.R --generator GEMINIPRO
Rscript CODE/RACE/RQ3/sensitivity_race.R     --generator GEMINIPRO

Additional experiments

A second launcher runs the extra experiments, using the same --attr / --generator convention. --plots-only (E1, gc) redraws the figures from the already-saved CSVs without recomputing any decomposition.

python CODE/run_experiments.py --list
python CODE/run_experiments.py e1 --attr GENDER --generator CHATGPT4
python CODE/run_experiments.py gc --attr GENDER
python CODE/run_experiments.py all --attr GENDER          # runs e1, gc

all runs the data-reusing experiments e1 and gc. E3 is run explicitly, because it generates a corpus that must then be scored with run_sa.py before E1 or gc can consume it.

8. E3 — Template generator (deterministic control)

What the script does. Builds the deterministic TEMPLATE generator introduced in Step 3. Each sentence is a factual, sentiment-neutral description of the structured record followed by a tone clause that depends only on the context U (three fixed tone variants per U band, selected deterministically by row index).

The output is byte-compatible with the LLM generators and is written to DATASET/<ATTR>/TEMPLATE/GENERATED_SENTENCES/, with the matching COMPLETE_DATASET assembled automatically.

Run commands

python CODE/run_experiments.py e3 --attr GENDER
python CODE/EXPERIMENTS/E3_template_generator.py --attr all

Then score and analyse TEMPLATE like any other generator:

python CODE/run_sa.py --attr GENDER --generator TEMPLATE --model all
python CODE/run_experiments.py e1 --attr GENDER --generator TEMPLATE

9. E1 — DAG sensitivity

What the script does. For each scored model, E1 pools all scenarios and re-runs the decomposition under a family of alternative causal graphs, then reports, per model, whether each effect's sign verdict survives the perturbation:

• GENDER {G0, G1, G2, G3}: baseline; drop a mediator (Employment); drop the confounder Z (Age); reclassify the mediator as a confounder.
• RACE {G0, G2}: baseline; drop the confounder (single mediator, so only the confounder can be perturbed).

Each component (TV, DE, IE, SE) gets a 95% CI (value ± 1.96·sd) and a sign verdict in {+, -, 0}; a cell is robust when its verdict matches the baseline graph G0. TV is graph-invariant and serves as a built-in sanity check.

Outputs go to RESULT/<ATTR>/<GENERATOR>/E1_dag_sensitivity/: e1_estimates_long.csv, e1_verdict_matrix.csv, e1_robustness_summary.csv, and e1_dag_estimates.pdf.

Run commands

python CODE/run_experiments.py e1 --attr GENDER --generator CHATGPT4
python CODE/run_experiments.py e1 --attr RACE   --generator CHATGPT4

10. Generator comparison (gc)

What the script does. Runs the baseline decomposition (the same TV / DE / IE / SE as RQ2) on every generator's scored corpus for the SA models they share, and compares the point estimates and sign verdicts side by side. TEMPLATE is the control for the generation step: its tone is independent of the protected attribute given the context, so it adds no attribute-correlated content of its own (see the paper, RQ0).

Outputs go to RESULT/<ATTR>/GENERATOR_COMPARISON/, split into AGGREGATE/ (pooled over scenarios) and PER_SCENARIO/. Each folder holds a wide table with the per-generator value / CI / verdict plus a concordance flag, a tidy long CSV, and an estimates ± 95% CI figure (PDF).

Run commands

python CODE/run_experiments.py gc --attr GENDER
python CODE/run_experiments.py gc --attr RACE

11. Proxy and age-sensitivity analyses

Two standalone scripts support the content-level analyses reported in the paper (RQ0 proxy analysis; RQ2 discussion of the spurious component). Both are self-contained and only read the already-generated datasets.

proxy_analysis.py — checks whether the protected attribute is recoverable from the generated sentences once all attribute-bearing terms are stripped (TF-IDF + logistic regression, 5-fold stratified CV), reported separately for the scenarios where the X–U correlation is off (accuracy expected at chance) and where it is injected by design (above-chance accuracy driven by the context-tone vocabulary).

python CODE/EXPERIMENTS/ADDITIONAL_ANALYSIS/proxy_analysis.py gender DATASET/GENDER/CHATGPT4/COMPLETE_DATASET/generated_sentences_gender_complete.csv
python CODE/EXPERIMENTS/ADDITIONAL_ANALYSIS/proxy_analysis.py race   DATASET/RACE/CHATGPT4/COMPLETE_DATASET/generated_sentences_race_complete.csv

age_sensitivity.py — measures the scorers' sensitivity to the age mention (OLS of the score on Age with covariate and scenario controls, on the scenarios where the spurious coefficients are inactive), and combines it with the injected age gap between protected groups to bound the implied spurious effect.

python CODE/EXPERIMENTS/ADDITIONAL_ANALYSIS/age_sensitivity.py gender RESULT/GENDER/<GENERATOR>/RQ2/SA/<model_name>/<scores_file>.csv
python CODE/EXPERIMENTS/ADDITIONAL_ANALYSIS/age_sensitivity.py race   RESULT/RACE/<GENERATOR>/RQ2/SA/<model_name>/<scores_file>.csv