Is AI bias all that new?

We've heard that AI (actually machine learning) can be biased unintentionally when, for example, the Uber matching algorithm makes women less money, or an inability to tell people with dark skin hues apart. The problem is often laid at the feet of incorrect sampling during the "learning" phase of machine learning. But sampling bias has emerged long before machine learning. Arguably, sampling bias is the biggest problem in any sort of statistical analysis and prediction ever conducted. Sometimes machine learning is blamed for sampling bias even if when no machine learning is involved.

Machine learning doesn't fix sampling bias because it doesn't aim to and it's not asked to. It does present an opportunity for device makers to think about sampling bias, for regulators to think about regulations that mitigate it, and for academics to disseminate clear guidelines on how to minimize it in medicine.

The key to all of these is the ability to measure sampling bias and reproducibility. That is, to reproduce a machine learning algorithm exactly, in multiple settings on different data sets, and produce comparable result that before choosing to adopt a machine learning algorithm for a particular settings.

Addressing Bias in Artificial Intelligence in Health Care