Introduction
In May 2016, ProPublica found that risk scores used nationwide to predict whether a defendant will commit a crime in the future are biased against Black people.
In a study of 7,000 defendants, their risk scores, and their actual recidivism rates, “[w]hite defendants were mislabeled as low risk more often than [B]lack defendants.”
For-profit companies like Equivant survey and collect data on defendants and then generate these biased risk scores.
Like most developers, their goal is to create a more efficient, productive system that prevents the introduction of human errors. “The trick, of course, is to make sure the computer gets it right.”
In 2019, Ziad Obermeyer—a professor at the University of California, Berkeley’s School of Public Health—and his team were looking into how algorithms inform healthcare management in large hospitals.
Their research revealed not only a problem in the healthcare industry but the tip of a systemically racist iceberg. An algorithm that was widely used by hospitals in the United States to help allocate healthcare to the patients visiting the hospital was guilty of “systematically discriminating against [B]lack people.”
The data from the hospital Obermeyer and his team studied showed that the people who self-identified as Black “were generally assigned lower risk scores than equally sick white people.”
Black patients, though equally sick, were wrongly considered to be in less urgent or immediate need of care than white patients.
The information and data that are collected on people can be twisted and used in discriminatory ways. Now more than ever, “the amount and variety of data that is collected from individuals has increased exponentially, ranging from structured numeric data to unstructured text documents such as email, video, audio and financial transactions.”
Companies use, sell, and share this information.
Further, law enforcement buys these data to build massive and discriminatory police surveillance networks.
All these personal datasets are summarized using a collection of methods identified by scholars as “Big Data analytics,”
and they can be used to inform companies and institutions on whether to approve a loan, grant parole, or deny a job application, among other things.
With access to Big Data, machine learning comes in to help uncover consumer trends and patterns with the help of decisionmaking algorithms.
Businesses find it helpful when these algorithms categorize and recognize patterns in the data that they can use.
Although the concepts of information and patterns on their own give the impression of impartiality, bias and racism thrive off Big Data analysts sharing and selling data.
What, then, is being done about this? Only recently has the United States embarked on the journey of building privacy regulations and data-protection laws to protect the people who use varied technologies, social media, and websites.
The United States has seen enormous transformation in terms of privacy regulation and steps taken to combat the potential dangers inherent in a society that is interwoven with the online world.
States have been stitching together the first wave of defenses against privacy infringements on a state-by-state basis.
Five states have taken necessary steps to strengthen their data privacy laws. These states have created seemingly more robust and comprehensive legislation that establishes a standard for consumer privacy. This legislation is already being enforced in these five states: California, Colorado, Connecticut, Utah, and Virginia.
California led the way on these data privacy and consumer laws with the California Consumer Privacy Act (CCPA), and the other four followed suit, even using much of the same verbiage as the CCPA.
Much of this language addresses companies that collect data, mandating full disclosure of what data is being taken and whether it is being sold.
Additionally, these laws mandate opt-out provisions in an effort to allow people to take further individual control over whether their data can be sold or accessed.
Data and consumer privacy concerns are rapidly growing: At least thirty-five states and the District of Columbia introduced or considered almost two hundred consumer privacy bills in 2022.
As these data privacy concerns and protections morph and transform so rapidly, companies and organizations are keeping a close eye on quickly evolving state regulations to stay on top of how they would need to respond to such shifts. Law enforcement has already begun using data-driven predictive models to zero in on areas and communities likely to be involved in criminal activities.
Many of these data, which reflect preexisting biased arrest patterns, perpetuate the problem. Some policing in departments such as the Los Angeles Police Department (LAPD) has moved to “Big Data Policing,” also called “data-informed community-focused policing” (DICFP).
Under this policy, law enforcement even coordinates directly with tech firms to surveil a person’s presence online (social media postings), to investigate crimes, and to monitor what it would deem potential threats.
Before many of the consumer privacy regulations, tech companies were under little to no obligation to inform their users about how they were sharing their users’ data or the ways their users’ actions were being monitored.
The emergence of the opt-out provision, specifically, returned some degree of agency to consumers over their privacy permissions and whether they allow a company to share or sell their data. In 2012, a story emerged detailing how Target was able to predict people’s pregnancies before they had so much as told their families.
These predictions were possible thanks to a statistician, “predictive analytics,” and unfettered access to consumer data.
Consumer data privacy and the right to opt out emerged after such events, and those who seek it out may exercise some agency to avoid similar situations. At least, that is the perception. Virtually all the enhanced-privacy and consumer protection regulations relevant here were passed within the previous three years, and countless more are inevitably on the horizon.
In the flurry of new laws both passed and upcoming, no one has thoroughly evaluated how effective these regulations are at avoiding these potential avenues of racism and bias. The opt-out provisions found within each of the new regulations contain “antidiscrimination” sections, but the language therein is thin and leaves many questions unanswered.
Is allowing for opt-out provisions and providing antidiscrimination language really benefitting diverse and marginalized communities? Or is it merely cementing the position of surveillance and tracking in our society while just making it transparently known that this is the status quo?
These new privacy laws’ variety and novelty raise questions about their effectiveness and the impact that they actually have. Since machine learning of people’s behaviors and preferences leads to wide-scale algorithmic bias, certain consumers opting out has also impacted the machine learning’s algorithmic process. That is to say, there is algorithmic bias based on who does opt out versus who does not. Access to consumer data can create discriminatory and unequal treatment, which may be exacerbated by disparities in participation in opt-out provisions, increasing the vulnerability of populations less aware of or less educated about the potential dangers. It is crucial that the United States implement a more robust regulatory system regarding its opt-out provisions to protect those who are most vulnerable in the digital world.
This Note starts in Part I with a discussion of the history of discrimination enabled by a lack of data privacy. Part I then turns to state-specific privacy regulations, providing a general overview of the key rights found in these regulations and discussing the regulations’ strengths. Part II looks at the laws as they are applied and breaks down the ways that the regulations may generate discrimination based on who decides to opt out. Part III addresses potential remedies in the form of a national privacy framework; mandated opt-in provisions in place of opt-out provisions; and altered presentation of the existing opt-out website pop-ups to make them both easy to understand and unavoidable by consumers.