health life sciences medicine
We introduce Q-Pain, a dataset for assessing bias in medical QA in the context of pain management. We developed 55 medical question-answer pairs across five different types of pain management: each question includes a detailed patient-specific medical scenario ("vignette") designed to enable the substitution of multiple different racial and gender "profiles" and to evaluate whether bias is present when answering whether or not to prescribe medication. Along with the dataset, we propose a new framework, including a sample experimental design, to measure the potential bias present during medical decision-making for patients with particular demographic profiles. We demonstrate its use in assessing two reference QA systems, GPT-2 and GPT-3, selected for their documented ability to answer questions given only a few examples. We hope that our dataset and framework will be used to assess bias across a wide variety of pain management QA systems.
Not updated
Creative Commons Attribution-ShareAlike 4.0 International Public License
https://doi.org/10.13026/2tdv-hj07
See all datasets managed by PhysioNet.
https://physionet.org/about/#contact_us
Q-Pain: A Question Answering Dataset to Measure Social Bias in Pain Management was accessed on DATE from https://registry.opendata.aws/q-pain.
arn:aws:s3:::physionet-open/q-pain/us-east-1aws s3 ls --no-sign-request s3://physionet-open/q-pain/