The Bias Report

 

Audit Date: 30 Jan 2021
Data Audited: 7214 rows
Attributes Audited: race, sex, age_cat
Audit Goal(s): Equal Parity - Ensure all protected groups are have equal representation in the selected set.
Proportional Parity - Ensure all protected groups are selected proportional to their percentage of the population.
False Positive Rate Parity - Ensure all protected groups have the same false positive rates as the reference group).
False Discovery Rate Parity - Ensure all protected groups have equally proportional false positives within the selected set (compared to the reference group).
False Negative Rate Parity - Ensure all protected groups have the same false negative rates (as the reference group).
False Omission Rate Parity - Ensure all protected groups have equally proportional false negatives within the non-selected set (compared to the reference group).
Reference Groups: Automatically select, for each bias metric, the group on each attribute that has the lower value, to be used as baseline to calculate relative disparities in this audit.
Fairness Threshold: 80%. If disparity for a group is within 80% and 125% of the value of the reference group on a group metric (e.g. False Positive Rate), this audit will pass.

 


 

Audit Results:

  1. Summary

  2. Details by Fairness Measures

  3. Details by Protected Attributes

  4. Bias Metrics Values

  5. Base Metrics Calculated for Each Group

 


 

Audit Results: Summary

Equal Parity - Ensure all protected groups are have equal representation in the selected set. Failed Details
Proportional Parity - Ensure all protected groups are selected proportional to their percentage of the population. Failed Details
False Positive Rate Parity - Ensure all protected groups have the same false positive rates as the reference group). Failed Details
False Discovery Rate Parity - Ensure all protected groups have equally proportional false positives within the selected set (compared to the reference group). Failed Details
False Negative Rate Parity - Ensure all protected groups have the same false negative rates (as the reference group). Failed Details
False Omission Rate Parity - Ensure all protected groups have equally proportional false negatives within the non-selected set (compared to the reference group). Failed Details

 


 

Audit Results: Details by Fairness Measures

 

Equal Parity: Failed

What is it? When does it matter? Which groups failed the audit:
This criteria considers an attribute to have equal parity is every group is equally represented in the selected set. For example, if race (with possible values of white, black, other) has equal parity, it implies that all three races are equally represented (33% each)in the selected/intervention set. If your desired outcome is to intervene equally on people from all races, then you care about this criteria. For race (with reference group as Asian)
   Caucasian with 106.75X Disparity
   Native American with 1.50X Disparity
   Hispanic with 23.75X Disparity
   African-American with 271.75X Disparity
   Other with 9.88X Disparity

For sex (with reference group as Female)
   Male with 4.61X Disparity

For age_cat (with reference group as Greater than 45)
   25 - 45 with 4.88X Disparity
   Less than 25 with 2.54X Disparity

 

Go to Top

 


 

Proportional Parity: Failed

What is it? When does it matter? Which groups failed the audit:
This criteria considers an attribute to have proportional parity if every group is represented proportionally to their share of the population. For example, if race with possible values of white, black, other being 50%, 30%, 20% of the population respectively) has proportional parity, it implies that all three races are represented in the same proportions (50%, 30%, 20%) in the selected set. If your desired outcome is to intervene proportionally on people from all races, then you care about this criteria. For race (with reference group as Other)
   Caucasian with 1.66X Disparity
   African-American with 2.81X Disparity
   Native American with 3.18X Disparity
   Hispanic with 1.42X Disparity

For age_cat (with reference group as Greater than 45)
   25 - 45 with 1.87X Disparity
   Less than 25 with 2.61X Disparity

 

Go to Top

 


 

False Positive Rate Parity: Failed

What is it? When does it matter? Which groups failed the audit:
This criteria considers an attribute to have False Positive parity if every group has the same False Positive Error Rate. For example, if race has false positive parity, it implies that all three races have the same False Positive Error Rate. If your desired outcome is to make false positive errors equally on people from all races, then you care about this criteria. This is important in cases where your intervention is punitive and has a risk of adverse outcomes for individuals. Using this criteria allows you to make sure that you are not making false positive mistakes about any single group disproportionately. For race (with reference group as Asian)
   Caucasian with 2.70X Disparity
   African-American with 5.16X Disparity
   Hispanic with 2.47X Disparity
   Native American with 4.31X Disparity
   Other with 1.70X Disparity

For age_cat (with reference group as Greater than 45)
   Less than 25 with 3.22X Disparity
   25 - 45 with 1.99X Disparity

 

Go to Top

 

 


 

 

False Discovery Rate Parity: Failed

What is it? When does it matter? Which groups failed the audit:
This criteria considers an attribute to have False Discovery Rate parity if every group has the same False Discovery Error Rate. For example, if race has false discovery parity, it implies that all three races have the same False Discvery Error Rate. If your desired outcome is to make false positive errors equally on people from all races, then you care about this criteria. This is important in cases where your intervention is punitive and can hurt individuals and where you are selecting a very small group for interventions. For race (with reference group as Asian)
   Other with 1.82X Disparity
   Caucasian with 1.63X Disparity
   Hispanic with 1.83X Disparity
   African-American with 1.48X Disparity

For sex (with reference group as Male)
   Female with 1.34X Disparity

For age_cat (with reference group as Less than 25)
   Greater than 45 with 1.27X Disparity

 

Go to Top

 

 


 

 

False Negative Rate Parity: Failed

What is it? When does it matter? Which groups failed the audit:
This criteria considers an attribute to have False Negative parity if every group has the same False Negative Error Rate. For example, if race has false negative parity, it implies that all three races have the same False Negative Error Rate. If your desired outcome is to make false negative errors equally on people from all races, then you care about this criteria. This is important in cases where your intervention is assistive (providing helpful social services for example) and missing an individual could lead to adverse outcomes for them. Using this criteria allows you to make sure that you’re not missing people from certain groups disproportionately. For race (with reference group as Native American)
   Hispanic with 5.56X Disparity
   Other with 6.77X Disparity
   African-American with 2.80X Disparity
   Caucasian with 4.77X Disparity
   Asian with 3.33X Disparity

For age_cat (with reference group as Less than 25)
   Greater than 45 with 2.20X Disparity
   25 - 45 with 1.44X Disparity

 

Go to Top

 

 


 

 

False Omission Rate Parity: Failed

What is it? When does it matter? Which groups failed the audit:
This criteria considers an attribute to have False Omission Rate parity if every group has the same False Omission Error Rate. For example, if race has false omission parity, it implies that all three races have the same False Omission Error Rate. If your desired outcome is to make false negative errors equally on people from all races, then you care about this criteria. This is important in cases where your intervention is assistive (providing help social services for example) and missing an individual could lead to adverse outcomes for them , and where you are selecting a very small group for interventions. Using this criteria allows you to make sure that you’re not missing people from certain groups disproportionately. For race (with reference group as Asian)
   Hispanic with 2.31X Disparity
   Other with 2.42X Disparity
   Native American with 1.33X Disparity
   Caucasian with 2.31X Disparity
   African-American with 2.80X Disparity

For sex (with reference group as Female)
   Male with 1.36X Disparity

For age_cat (with reference group as Greater than 45)
   Less than 25 with 1.76X Disparity
   25 - 45 with 1.34X Disparity

 

Go to Top

 


Audit Results: Details by Protected Attributes

 

race

 

Attribute Value Equal Parity Proportional Parity False Discovery Rate Parity False Positive Rate Parity False Omission Rate Parity False Negative Rate Parity
African-American African-American African-American African-American African-American African-American African-American
Asian Ref Asian Ref Ref Ref Asian
Caucasian Caucasian Caucasian Caucasian Caucasian Caucasian Caucasian
Hispanic Hispanic Hispanic Hispanic Hispanic Hispanic Hispanic
Native American Native American Native American Native American Native American Native American Ref
Other Other Ref Other Other Other Other

Go to Top

 

 

sex

 

Attribute Value Equal Parity Proportional Parity False Discovery Rate Parity False Positive Rate Parity False Omission Rate Parity False Negative Rate Parity
Female Ref Ref Female Ref Ref Female
Male Male Male Ref Male Male Ref

Go to Top

 

 

age_cat

 

Attribute Value Equal Parity Proportional Parity False Discovery Rate Parity False Positive Rate Parity False Omission Rate Parity False Negative Rate Parity
25 - 45 25 - 45 25 - 45 25 - 45 25 - 45 25 - 45 25 - 45
Greater than 45 Ref Ref Greater than 45 Ref Ref Greater than 45
Less than 25 Less than 25 Less than 25 Ref Less than 25 Less than 25 Ref

Go to Top

 

 


Audit Results: Bias Metrics Values

 

race

 

Attribute Value Predicted Positive Rate Disparity Predicted Positive Group Rate Disparity False Discovery Rate Disparity False Positive Rate Disparity False Omission Rate Disparity False Negative Rate Disparity
African-American 271.75 2.81 1.48 5.16 2.8 2.8
Asian 1.0 1.19 1.0 1.0 1.0 3.33
Caucasian 106.75 1.66 1.63 2.7 2.31 4.77
Hispanic 23.75 1.42 1.83 2.47 2.31 5.56
Native American 1.5 3.18 1.0 4.31 1.33 1.0
Other 9.88 1.0 1.82 1.7 2.42 6.77

Go to Previous

Go to Top

 

 

sex

 

Attribute Value Predicted Positive Rate Disparity Predicted Positive Group Rate Disparity False Discovery Rate Disparity False Positive Rate Disparity False Omission Rate Disparity False Negative Rate Disparity
Female 1.0 1.0 1.34 1.0 1.0 1.06
Male 4.61 1.11 1.0 1.01 1.36 1.0

Go to Previous

Go to Top

 

 

age_cat

 

Attribute Value Predicted Positive Rate Disparity Predicted Positive Group Rate Disparity False Discovery Rate Disparity False Positive Rate Disparity False Omission Rate Disparity False Negative Rate Disparity
25 - 45 4.88 1.87 1.07 1.99 1.34 1.44
Greater than 45 1.0 1.0 1.27 1.0 1.0 2.2
Less than 25 2.54 2.61 1.0 3.22 1.76 1.0

Go to Previous

Go to Top

 

 


Audit Results: Group Metrics Values

 

race

 

Attribute Value Group Size Ratio Predicted Positive Rate Predicted Positive Group Rate False Discovery Rate False Positive Rate False Omission Rate False Negative Rate
African-American 0.51 0.66 0.59 0.37 0.45 0.35 0.28
Asian 0 0.0 0.25 0.25 0.09 0.12 0.33
Caucasian 0.34 0.26 0.35 0.41 0.23 0.29 0.48
Hispanic 0.09 0.06 0.3 0.46 0.21 0.29 0.56
Native American 0 0.0 0.67 0.25 0.38 0.17 0.1
Other 0.05 0.02 0.21 0.46 0.15 0.3 0.68

Go to Previous

Go to Top

 

 

sex

 

Attribute Value Group Size Ratio Predicted Positive Rate Predicted Positive Group Rate False Discovery Rate False Positive Rate False Omission Rate False Negative Rate
Female 0.19 0.18 0.42 0.49 0.32 0.24 0.39
Male 0.81 0.82 0.47 0.36 0.32 0.33 0.37

Go to Previous

Go to Top

 

 

age_cat

 

Attribute Value Group Size Ratio Predicted Positive Rate Predicted Positive Group Rate False Discovery Rate False Positive Rate False Omission Rate False Negative Rate
25 - 45 0.57 0.58 0.47 0.39 0.33 0.32 0.37
Greater than 45 0.22 0.12 0.25 0.46 0.17 0.24 0.57
Less than 25 0.21 0.3 0.65 0.36 0.54 0.42 0.26

Go to Previous

Go to Top