This week’s read, Weapons of Math Destruction, delved into the topic of how algorithms can cause harm. It’s a topic we touched on with Jules’s book report on “Algorithms of Oppression,” and one that’s critically important in today’s world. As O’Neil tells us, these algorithms, which she terms Weapons of Math Destruction or WMDs, are being used to manage our lives. Some people think that math is unbiased, free of human error and able to manage things much more fairly than a human arbiter could. The truth is more complicated than that.
What Are These Algorithms?
Merriam-Webster defines an algorithm as “a procedure for solving a mathematical problem…in a finite number of steps that frequently involves repetition of an operation.” For our purposes, an algorithm is a complex system of equations that makes decisions about data. O’Neil gives us the example of an algorithm being used (poorly) to “weed out bad teachers.” An article by a different O’Neil talks about an algorithm that denied an unhoused woman access to housing. Algorithms also read resumes, make loan decisions…and of course, facilitate our searches.
What’s Wrong with Them?
Why are these algorithms failing so badly? There are several reasons. One, algorithms are related to surveillance capitalism and the selling of data. O’Neil outlines the connection for us: mathematicians and statisticians study the data we produce online to predict our desires, spending power, and to calculate our potential. Potential for what? To be students, or to perform certain jobs…or to be criminals. Of course, these predictions are based on data produced by a flawed, discriminatory society. This leads to yet more discrimination in the digital world, creating digital redlining (which then extends over into the physical.) And, of course, because money is involved (it is surveillance capitalism after all), corporate interests skew results.
O’Neil also gets into some more technical reasons why algorithms fail. To be statistically sound, these models would have to contain large numbers of samples, which is what statisticians count on to balance out anomalies. In most of these cases though, such as the one with the teachers, the algorithms are only analyzing about 30 pieces of data! Another thing is that systems require feedback to determine if they’re predicting things correctly…but these algorithms are not getting feedback. They are creating and enforcing their own truths.
Can We Just Stop Using Math (Models)?
As much as I hate math, I understand that we can’t stop using it, and in today’s world, we also can’t stop using algorithms. They’re everywhere, inescapable. And hey, anyone who finds this blog post outside of our class will have found it via algorithm. So what are we to do? O’Neil mentions a sort of “Hippocratic oath” that data scientists should take…but that relies on data scientists being concerned with morals. What if they’re not? We need laws in place to regulate these things. Of course, I feel like I’ve been saying that over and over in these posts…with each reading, the lack of internet privacy laws and data protections becomes more and more glaring.
Leave a Reply