• Eric Loomis pled guilty to driving a stolen vehicle & fleeing police
  • Before sentencing, the court looked at an ML assessment (COMPAS) of his likelihood to reoffend. Loomis was rated a high risk
  • COMPAS was not trained for sentencing, but for resource allocation after release
  • COMPAS is biased in favor of whites. Loomis is white
  • The court sentenced him to 8 years in prison, in part because of his high risk rating


  • Houston SD purchased an ML tool (EVAAS) to evaluate teachers' effectiveness. The system is proprietary
  • Houston fired 25% of teachers with ineffective scores, based solely on their scores
  • There is no process for teachers to inspect, challenge, or correct their data or scores
  • Known errors cannot be corrected because of how the model normalizes: fixing one teacher's data has the potential to change all other teachers' scores


  • MSU uses cookies, ML, and third party datasets to score possible students who visit their site
  • The scores favor those:
    • likely to accept an admissions offer to MSU
    • applying to majors the school is already strong in
    • wealthy out-of-state students
  • The scores are used to determine who to focus recruitment efforts on, not who to accept


What's Fair?

Mariah A. Knowles

Tue, Dec 6, 2022

image: Diana Ejaita for Vox

When we say something is fair...

Fairness is a conclusion

Fair amounts to Good, Justified, Okay, Fine, Morally Acceptable, ...

Fair amounts to 👍

Fair is a thick evaluation, colloquially

Fair amounts to 👍 + some set of morally relevant criteria


Problem 1: Fair covers a range of moreorless related concepts, some competing

Problem 2: It's easy to talk past each other

  1. Technology creates risks
  2. We have responsibilities w.r.t. those risks
  3. One type of risk is:
    People will be subject to an environment
    that runs afoul
    of fair terms of social cooperation

Some Fair Terms

  • Malice
  • Respect for Individuals
  • Acceptable Social Arrangements
  • Deception
  • Acceptable Judgment

We ought not hold malice towards other groups

We ought to respect others as individuals

Power and Freedom ought to be distributed justly
Harms and Goods ought to be distributed "equally"

We ought not deceive others

We ought not judge others on things outside their control
We ought not judge others on things that bear no relevance to the outcome