How Facebook’s Advertising Algorithms Can Discriminate By Race and Ethnicity

Jinyan Zang

Abstract

Over the last 5 years, Facebook has faced repeated criticism and lawsuits over the potential for discrimination on its ad platform. In July 2020, advocacy groups focused on issues of misinformation and civil rights organized a high-profile boycott of Facebook’s advertising platform that successfully pressured more than a thousand major corporations to stop advertising on Facebook for the month. Facebook responded by releasing a civil rights audit and announcing the removal of much-criticized multicultural affinity groups, such as “African American (US)”, “Asian American (US)”, and “Hispanic (US – All)”, as ad targeting options. This study examines whether Facebook has gone far enough to prevent discrimination on its advertising platform. I collected data on Facebook’s ad platform in January 2020 and in January 2021. I compared the racial and ethnic breakdown of the old multicultural affinity groups with similar-sounding cultural interest groups such as “African-American Culture”, “Asian American Culture”, and “Hispanic American Culture” that were still usable by advertisers in 2021. I also used a set theory approach to study racial and ethnic biases in Facebook’s Lookalike Audience and Special Ad Audience algorithms in both time periods.

Results summary: I found that in 2021 Facebook’s “African-American Culture” ad targeting option contained 75% fewer White users than the old “African American (US)” option they removed in the previous year. Facebook’s tools to help advertisers find users similar to their existing customers exhibited bias toward including more African-Americans or Whites depending on which racial group was dominant in an advertiser’s customer list, and this was true for the Lookalike Audience tool, as well as the Special Ad Audience tool that Facebook created to explicitly not use sensitive demographic attributes when finding similar users. The degree of bias towards including more African-Americans or Whites in a Lookalike or Special Ad audience was larger when using customer lists of individuals with racially stereotypical names or ZIP codes as the basis for each tool. There was also significant bias towards Asians in Lookalike audiences based on Asians, reaching up to 100% Asian in one case. Finally, Lookalike audiences based on Hispanics overrepresented Hispanics versus Non-Hispanics. This study shows that in 2021, advertisers are able to use Facebook’s ad platform to discriminate by race and ethnicity by using cultural interest groups as targeting options or by using the Lookalike and Special Ad Audience tools. It also provides evidence that “fairness through unawareness”, the idea that discrimination is prevented by eliminating the use of protected class variables or close proxies in a model, does not reduce the potential for algorithmic bias.

Background

The Rise of Digital Advertising and Microtargeting

In 2019, digital advertising spending ($129 billion) in the US eclipsed traditional advertising ($109 billion) for the first time Facebook accounted for 22% of all digital ad spending, second only to Google’s 37% market share [1]. Regulatory agencies, advocacy groups, the media, and researchers are increasingly focused on the need to prevent advertising discrimination on the country’s second most popular digital advertising platform.

Facebook’s advertising platform provides multiple targeting tools to help an advertiser “micro target” only the users they want. These tools are the focus of this study.

“Detailed Targeting” options allow an advertiser to target a prepackaged group of Facebook users who share common attributes based on Facebook’s data analysis of the ads they click, the pages they engage with, the activities they conduct on its websites, and other data. “Detailed Targeting” options are organized into three categories: demographics, interests, and behaviors (Figure 1).

Figure 1. Adding Detailed Targeting Option to a Facebook Advertising Campaign.
“Custom Audiences” allow an advertiser to either upload a contact list of customers or other individuals for Facebook to target or create an audience by integrating Facebook’s trackers on their websites or apps. When an advertiser uploads a customer list, Facebook then matches data fields such as email, phone number, first name, last name, city, state, country, ZIP code, date of birth, gender, and more with the data it has on its users. Choosing a Custom audience means targeting ads only to Facebook users that match an advertiser’s customer list (Figure 2). Facebook can also create Custom audiences based on which Facebook users have interacted with an advertiser’s content on the Facebook platform by an act such as commenting on a video, liking an Instagram post, or attending a Facebook event (Figure 3).

Figure 2. Identifiers Used to Match an Advertiser’s Customer List with Users on Facebook.

Figure 3. Sources for Custom Audience on Facebook.
“Lookalike Audiences” allow an advertiser to reach people who resemble members of a designated source audience created through the Custom Audiences tool. According to Facebook, “You choose a source audience, and we identify the common qualities of the people in it. Then we find people like them, using your selected location and desired audience size.” The advertiser can choose one or multiple countries as the “Audience Location”, and Facebook will find the 1-10% of the population of Facebook users in those countries that are most similar to the advertiser’s source audience (Figure 4).

Figure 4. Creating a Lookalike Audience.
“Special Ad Audiences” allow an advertiser to create a Lookalike Audience that finds people similar to a source audience “in online behavior without considering things like age, gender or ZIP code”[2] specifically for ads in the categories of housing, employment, and credit, which are regulated by anti-discrimination laws (Figure 5). The settings for creating a Special Ad Audience are similar to the Lookalike Audience settings. The advertiser chooses a source audience, then selects one or multiple countries as the “Audience Location”, and finally asks Facebook to find the 1-10% of the population of Facebook users in those countries who are most similar to the advertiser’s source audience (Figure 6).

Figure 5. Special Ad Audience Tool.

Figure 6. Creating a Special Ad Audience.

An advertiser can use the tools offered on Facebook’s ad platform to define a desired intersection of user attributes by using Facebook’s Audience selection tool (Figure 7). First, the advertiser can choose one or multiple Custom, Lookalike, or Special Ad audiences to “include” as potential targets. Advertisers can also choose one or multiple Custom or Lookalike audiences to exclude as potential targets, though they are not able to choose to exclude a Special Ad audience. Next, in terms of geographical targeting options, the advertiser can choose to limit the audience to individuals currently living in, recently located in, or traveling to a particular country, state, city, ZIP code, media market, or Congressional District. Another option is to drop a pin on the map and target all individuals within f 1 to 50 miles of the pin’s location. The advertiser also has options to target by age, gender, and language. The advertiser can choose to add one or more of the predefined “Detailed Targeting” options based on demographics, interests, and behaviors created by Facebook to include or exclude as criteria for their target audience. There are also additional options for an advertiser to include or exclude users who have connections with the advertiser’s Facebook pages, apps, or events.

Figure 7. Facebook’s Audience selection tool.

Digital advertising platforms like Facebook give advertisers abilities to target individuals, offering options that traditional advertising does not provide or cannot offer in one place.

First, with traditional print or broadcast advertising, the advertiser will reach anyone who reads the newspaper or watches the TV show that displays the ad. On Facebook, the advertiser can target specific individuals they want to reach through Facebook’s Custom Audiences or match a very specific intersection of targeting criteria, combining demographics, interests, behaviors, and more (Figure 7).

Second, with physical mail or email advertising, the advertiser can usually only reach those whose physical or email address they already have. On Facebook, however, the advertiser can potentially target millions of new individuals similar to existing customers without needing to know contact information for this new audience. Facebook offers this capability through its Lookalike Audiences or Special Ad Audiences tools.

Facebook and Race

The increased targeting capabilities of Facebook’s advertising platforms also present new risks for advertising to have a discriminatory result.

Given this potential for discrimination, it is important to note that Facebook does not ask users to indicate their race. However, until August 11, 2020, the Facebook Ads platform did infer a user’s “Multicultural Affinity” based on their behavior on Facebook and how similar it is to others of the same affinity group. Based on this inference Facebook offered advertisers the ability to target by Multicultural Affinity classifications for “African American”, “Asian American”, and “Hispanic”, with the following Hispanic sub-categories: “Hispanic – Bilingual”, “Hispanic – Spanish Dominant”, and “Hispanic – English Dominant”.

According to Facebook:

The word “affinity” can generally be defined as a relationship like a marriage, as a natural liking, and as a similarity of characteristics. We are using the term “Multicultural Affinity” to describe the quality of people who are interested in and likely to respond well to multicultural content. What we are referring to in these affinity groups is not their genetic makeup, but their affinity to the cultures they are interested in…. The Facebook multicultural targeting solution is based on affinity, not ethnicity. This provides advertisers with an opportunity to serve highly relevant ad content to affinity-based audiences [21].

While Facebook argued that these multicultural affinity classifications did not facilitate racial and ethnic discrimination by advertisers, it faced repeated criticism from journalists, researchers, civil rights groups, and law enforcement agencies over this issue.

Media Attention

In October 2016, ProPublica journalists Julia Angwin and Terry Parris Jr. found that “Facebook lets advertisers exclude users by race” [3]. Advertisers could target or exclude users based on Facebook’s “Ethnic Affinity” categories, which included African American, Asian American, and Hispanic. In order to demonstrate the possibility of discrimination, ProPublica ran its own housing-related ads on Facebook and used these categories to exclude users of a given ethnic affinity (Figure 8). Facebook responded that ethnic affinity categories existed as part of their “multicultural advertising” effort and that ethnic affinity was not the same as race but rather a membership category that Facebook created based on the pages and posts a user liked or engaged with on its website [3].

Figure 8. ProPublica’s example in October 2016 of using the “Ethnic Affinity” categories within Facebook’s Detailed Targeting options to potentially exclude minority users from seeing a housing-related ad [3].

One year after their initial reporting, ProPublica journalists reported in a November 2017 story, “Facebook (Still) Letting Housing Advertisers Exclude Users by Race” [4], that discrimination in housing advertising was still allowed on the platform. ProPublica was able to purchase housing-related ads that excluded users on the basis of multicultural affinity groups, as well as mothers of high school kids, people interested in wheelchair ramps, Jews, expats from Argentina, and Spanish speakers. These categories appear related to “protected classes” in anti-discrimination law. Facebook approved every ad proposed by ProPublica within three minutes. Facebook acknowledged that “This was a failure in our enforcement and we’re disappointed that we fell short of our commitments” [4]. ProPublica noted that Facebook had renamed “Ethnic Affinity” as “Multicultural Affinity” and moved it from the Demographics targeting options to the Behaviors list. Since Facebook also lets advertisers select which ZIP codes to target, ProPublica could also target ads at majority non-Hispanic White ZIP codes in Brooklyn. This demonstrated a practice ProPublica described as similar to “redlining” [4], a set of historical discriminatory practices used by landlords, brokers, and lenders to prevent African-Americans from moving to predominantly White neighborhoods. Redlining is now prohibited by the Fair Housing Act.

Academic Research

Academic researchers have also found discriminatory potential in Facebook’s other targeting tools, including Lookalike Audiences and Special Ad Audiences. A December 2019 study found that Lookalike and Special Ad audiences could be significantly biased by the demographics of the source audience [5]. For example, when the source audience was all women, the Lookalike audience was 96.1% women, and the Special Ad audience was 91.2% women. Similar relationships were also observed for racial groups. For example, a source audience that was 100% Black created a Lookalike audience which had a 61% overlap with a list of 900,000 Black voters and only a 16% overlap with a second list of 900,000 White voters in the same state. The corresponding Special Ad audience had a 62% overlap with the Black voter list and a 12% overlap with the White voter list. On the other hand, a Lookalike Audience based on a source audience that was 100% White had a much smaller overlap of 17% with the Black voter list and a 42% overlap with the White voter list. Similarly, the corresponding Special Ad Audience had a 10% overlap with the Black voter list and a higher 36% overlap with the White voter list.

Legal Actions Taken Against Facebook and Its Advertisers

Facebook’s policies and practices have been challenged in numerous lawsuits alleging illegaladvertising discrimination. On November 3, 2016, a class action lawsuit, Mobley v. Facebook, filed in U.S. District Court, argued that Facebook violated federal anti-discrimination laws for housing (Fair Housing Act) and employment (Civil Rights Act) by citing the ProPublica reporting [22, 23]. On March 27, 2018, a coalition of housing advocacy groups led by the National Fair Housing Alliance (NFHA) filed a lawsuit against Facebook for violating the Fair Housing Act by allowing advertisers to discriminate against legally protected groups such as mothers, the disabled, and Spanish-language speakers [24]. The NFHA conducted its own investigation of Facebook’s advertising platform and found that it could exclude individuals that Facebook classified as “Disabled American Veterans”, “moms of preschool kids”, and individuals interested in “English as a second language” [6]. On September 18, 2018, the ACLU, the Communications Workers of America (CWA), and the employment law firm Outten & Golden LLP filed charges with the Equal Employment Opportunity Commission (EEOC) against Facebook and ten major corporations that targeted ads for jobs to younger male Facebook users only, excluding all women and older users [7]. In a separate 2018 lawsuit filed by the Communications Workers of America vs. T-Mobile, Amazon, and 1,000 other large employers using Facebook ads, the plaintiffs allege that not only did the employers use Facebook’s prepackaged targeting options in a discriminatory manner, but they also used Facebook’s Lookalike Audience tool to target candidates demographically similar to their existing workforce in ways that marginalized older workers [8, 25].

Regulatory agencies and other law enforcement offices have also investigated and filed suit against Facebook for advertising discrimination. In 2016, Washington State’s Office of the Attorney General started a 20-month investigation of Facebook’s advertising platform. The investigators purchased ads that excluded protected categories of people from being targeted and found real-world examples of ads that did just that [10, 26, 27]. Washington Attorney General Bob Ferguson said, “Facebook’s advertising platform allowed unlawful discrimination on the basis of race, sexual orientation, disability and religion…That’s wrong, illegal, and unfair” [26].

On March 28, 2019, the U.S. Department of Housing and Urban Development (HUD) sued Facebook, alleging it “unlawfully discriminates based on race, color, national origin, religion, familial status, sex, and disability by restricting who can view housing-related ads on Facebook’s platforms and across the internet” and “mines extensive data about its users and then uses those data to determine which of its users view housing-related ads based, in part, on these protected characteristics” [9].

Political Controversy and Criticism from Civil Rights Advocacy Groups

Finally, while it is not unlawful in the United States to target political or news ads in a racially biased manner, the ability to use Facebook’s advertising platform to spread misinformation or inflame racial tensions generated high-profile political controversy in the 2016 and 2020 General Elections.

Researchers studying the 3,519 ads that Facebook shared with Congress as part of investigations into Russian interference with the 2016 presidential election found that many of the ads focused on Black identity issues such as police shootings, Black Lives Matter, and discrimination [28]. The most popular Facebook targeting options used by Russian actors included targeting users interested in Martin Luther King, African-American Civil Rights Movement, African-American history, Black Power, and related categories [15]. In fact, 17 ads used Facebook’s “African-American (US)” multicultural affinity group [15]. Another study found that across all the Russian-linked ads disclosed by Facebook, 52% had more than double the proportion of African-Americans in their target audience compared to Facebook’s US baseline [29]. Besides the Russians, other political actors may have also used Facebook’s advertising tools to target minority voters in controversial ways in 2016. For example, in October 2016, Trump Campaign Manager Brad Parscale announced that the Trump campaign would target an ad featuring opponent Hillary Clinton disparaging young African-Americans as “superpredators”, using Facebook’s advertising platform to reach “only the people we want to see it”, in order to suppress support for Clinton among African-American voters [30].

In 2020, similar controversies arose about misinformation by campaigns targeting minority voters on Facebook. For example, racial appeals and a focus on American social unrest during the summer of 2020 were part of misinformation campaigns by groups linked to Russia, Iran, and China on Facebook and Twitter [11, 12]. Domestic political actors also used these platforms to promote misleading or false ads [11, 13, 14]. In August 2020, The Washington Post reported that FreedomWorks, a conservative political advocacy group established by David and Charles Koch, spent $1,500 on paid Facebook posts using LeBron James’ picture and falsely implying that James had criticized mail-in voting [14]. These ads targeted voters in swing states with high concentrations of minority voters [14]. Other political groups were able to exploit Facebook’s fact-checking system to re-post and distribute nearly identical copies of ads that Facebook’s fact-checking partners helped remove [13]. A significant amount of misinformation also targeted Hispanics in 2020, particularly Hispanic voters in Florida [31, 32, 33, 34]. According to Jaime Longoria, an investigative researcher with First Draft News, one tactic was to draw on “anti-blackness” bias to promote the idea that Black individuals were “harassing” Latinos under the guise of activism. One viral video that was shared 180,000 times on Facebook showed two Black women harassing a Latino family celebrating at a party. The video falsely labeled the women as members of Black Lives Matter [31]. Other misleading ads labeled presidential candidate Joe Biden a “communista” or stated that vice presidential candidate Kamala Harris supported abortion up to the minutes before birth [31, 32]. A common problem was that Facebook’s fact-checking was not as robust or strongly enforced for Spanish-language ads and posts [33].

In response to the widespread misinformation on Facebook, often targeting minority users, on June 17, 2020, civil rights groups including the NAACP, the Anti-Defamation League, and Color of Change launched the “Stop Hate for Profit” campaign. The campaign pressured major corporate advertisers to stop advertising on Facebook for the month of July 2020 [35]. More than 1,000 major advertisers joined the boycott, including Microsoft, Starbucks, Unilever, and Target [17]. The organizers of the boycott recommended 10 actions for Facebook to take, including hiring a C-suite executive to review the company’s products for discrimination, hate, and bias; participating in a regular third-party audit on identity-based misinformation and hate; stopping the amplification of content with ties to hate, misinformation, or conspiracies; ending the exemption of politicians from fact-checking; and other changes [35]. The organizers of the boycott argued that the prevalence of hate speech, white supremacy activism, and misinformation on Facebook reflects the systemic racism in America today and that Facebook’s technologies and the company’s laid-back approach to moderation further reinforce racism [36].

Racial Discrimination by Humans

Outside of Facebook, racial discrimination and racial bias can occur in many different situations in everyday life, from applying for a job or a loan to trying to get a doctor’s appointment. In these cases, individuals or a group of individuals working within a larger organization make the biased decisions. Just as Facebook does not collect race directly as a variable from its users, discrimination by others in society may not rely on an explicit racial variable but rather on proxies for race.

One common proxy is an individual’s name. Different racial groups tend to have distinct first and/or last names [37, 38]. Field experiments have shown that individuals with racially affiliated names can experience discrimination [39, 40]. For example, in 2003, Bertrand and Mullainathan found that resumes with White-sounding first names received 50% more callbacks for interviews than resumes with Black-sounding first names [39]. In 2016, Kang, DeCelles, et al. found a similar interview callback gap of 40% for James (a White-sounding first name) versus Lamar (a Black-sounding first name) when both resumes appeared “White”, i.e. the resumes did not describe participation in ethnically affiliated groups, such as an African-American fraternity. They found no statistical difference in the callback rates when the resumes appeared “Black”. When resumes appeared “Asian”, Luke Zhang (White-sounding first name, Asian-sounding last name) received a statistically significant 83% more interview callbacks than Lei Zhang (Asian-sounding first and last name) [40]. A meta-analysis of these field experiments in 2017 found no change in African-American callback rates since 1989 but found a decline in discrimination against Latinos [41]. Other studies have looked at gaps in response rates for a housing inquiry, a mortgage application, a request for a doctor’s appointment, a request for help from a public official, and more [42, 43, 44].

An individual’s home address can also serve as a proxy in other forms of racial discrimination. Historical factors have resulted in high levels of racial segregation in many US cities [45]. Before the passage of the Fair Housing Act in 1968, lenders and the real estate industry commonly practiced “redlining”, where residents of certain minority neighborhoods were denied access or offered inadequate access to affordable mortgages, and minority renters and borrowers were often prevented from moving to White-dominant neighborhoods [45]. While the Fair Housing Act made redlining and housing discrimination illegal, disparities in access to housing and borrowing still exist between racial groups [45]. For example, a 2015 study in Baltimore found that race is the most statistically significant factor in predicting who gets a mortgage, and that White borrowers were far more likely to get loans relative to the population baseline than African-American borrowers [46]. Similar patterns exist for 61 metro areas around the country [47]. Hanson and Hawley found in an audit study across the 10 largest US cities that African-Americans inquiring about rental vacancies faced lower response rates from landlords than Whites in racially mixed neighborhoods, areas with rent above the median rent for the city, and neighborhoods close to the city center or first-ring suburbs [44]. According to research by Raj Chetty, Nathaniel Hendren, et al., these disparities contribute to “substantially lower rates of upward mobility and higher rates of downward mobility” for African-Americans relative to Whites, “leading to large income disparities that persist across generations” [48]. Chetty et al. found that fewer than 5% of Black children grew up in rich areas with a poverty rate below 10% while more than 63% of White children did [48]. Among researchers studying algorithmic bias, these findings have contributed to concerns over the inclusion of geographic variables such as ZIP codes as data inputs. Such variables may serve as proxies for race and contribute to algorithmic bias [49, 50].

The research summarized above documents ways that discrimination often occurs “offline”. This study examines how it can occur online on sites like Facebook. I test whether the algorithms used by Facebook to create Lookalike and Special Ad audiences generate higher degrees of racial and ethnic bias when the advertising tools are given source audiences composed of individuals with racially and ethnically stereotypical names and ZIP codes.

Discrimination by Algorithms and Anti-Discrimination Laws

The Civil Rights Movement and its legislative successes resulted in three major federal laws that forbid racial and other types of discrimination in housing (Fair Housing Act), employment (Civil Rights Act of 1964), and credit (Equal Credit Opportunity Act), including in advertisements for those sectors [49]. These laws provide a legal recourse for victims of discrimination and a potential threat of legal repercussions to discriminatory actors. These are the laws that the ACLU, HUD, and others argue that Facebook and advertisers on Facebook’s platform violated. However, as discrimination has moved from decisions made by humans to those by machines and algorithms, detecting discrimination and enforcing anti-discrimination laws has become more complicated.

Latanya Sweeney’s 2013 study on Google’s ad platform was an early, high-profile example of a digital audit study to detect discrimination. Sweeney found that statistically significantly more ads using Black-identifying first names contained the word “arrest” in the ad’s text than ads using White-identifying first names, with an adverse impact ratio of 77% for Reuters.com, which had ads served by Google, and an adverse impact ratio of 40% for Google.com search results [51]. Other instances of algorithmic bias have been found in facial recognition systems [52], online shopping [53], search engines [54, 55, 56, 57], job sites and hiring software [58, 59, 60], translation services [61], healthcare [62], and other systems.

Since much of the work on algorithmic discrimination has focused on the products of private sector companies, the legal concepts of “disparate treatment” and “disparate impact” established by Title VII of the 1964 Civil Rights Act are the primary applicable jurisprudence [63].

Disparate treatment focuses on intentional discrimination, which occurs when individuals are treated differently because of their protected class attribute, such as race, color, national origin, religion, sex, disability, or familial status [63]. There can be overt evidence of disparate treatment, as when a lender has a policy of a higher credit limit for older borrowers [64]. Or there can be comparative evidence of disparate treatment, as when two borrowers who are otherwise similar are treated differently by a lender on the basis of a protected class attribute [64].

Disparate impact occurs when there is a disproportionate burden in outcomes for a specific group. Showing intentionality is not required if a significant disparity can be demonstrated [63, 64]. The law uses a burden-shifting framework to decide whether a company is liable under disparate impact. First, a plaintiff must show evidence of disproportionate outcomes across demographic groups. The Equal Employment Opportunity Commission’s (EEOC) 80-20 Rule is often cited by algorithmic bias literature, although it primarily applies to labor issues [65, 66]. In response, a defendant corporation can try to show a “business necessity” to justify its decision-making process [63, 64]. For example, if a job requires lifting heavy supplies, a business may argue that business necessity requires it to consider a candidate’s ability to lift weights in its hiring process, even if that leads to hiring more male candidates. Finally, the plaintiff must show that there is a less discriminatory alternative that could meet the business necessity [63].

These concepts may be applied to the Facebook ad targeting tools. If an advertiser uses Facebook’s targeting options in an intentionally discriminatory way, such as creating age limits or gender criteria, or choosing multicultural affinity targeting options like “African-American (US),” that may be considered disparate treatment. If an advertiser has a predominantly White customer base, they might then use either Facebook’s Lookalike Audience or Special Ad Audience tools to create a new target audience of other users whom Facebook identifies as being similar to their existing customers. If the resulting target audience is racially biased by being predominantly White, under the disparate impact framework, it doesn’t matter whether the advertiser or Facebook intentionally meant to discriminate. Plaintiffs challenging this choice would need to show that Black and other minority users were impacted disproportionately by not being targeted by the advertiser relative to White peers. The advertiser could argue that it was a business necessity to use Facebook’s tools to find new ad audiences similar to their existing customers. Facebook might argue that business necessity required creating the best possible Lookalike or Special Ad audience to serve their advertising clients. Finally, the plaintiffs would need to show that there are less discriminatory alternatives that could still achieve the same business necessities without predominantly targeting more White users.

This study examines to what degree Facebook’s racially affiliated targeting options lead to disparate treatment by race and ethnicity. This study also examines to what degree Facebook’s Lookalike Audience and Special Ad Audience tools lead to disparate impact by race and ethnicity depending on the demographics of the source audience.

Facebook’s Response to Charges of Discrimination

After being repeatedly accused of discrimination by journalists, advocacy groups, and law enforcement agencies, over time Facebook has become more responsive in making changes to its advertising platform.

In response to the original ProPublica report from 2016 [3], Facebook announced efforts in February 2017 to icreate enforcement tools to disapprove ads that use multicultural affinity groups – the new name “Ethnic Affinity” – as targeting options when offering housing, employment, or credit opportunities. Facebook also required advertisers to self-certify that they are not discriminating [67].

On July 24, 2018, Facebook signed a legally binding agreement with the state of Washington to make changes to its advertising tools as a result of the 20-month investigation conducted by Washington Attorney General Bob Ferguson’s office [10]. Facebook agreed to pay $90,000 in costs and fees to the Attorney General’s Office and removed targeting options that may allow advertisers to exclude users on the basis of race, religion, sexual orientation, veteran status, and other protected classes from seeing ads for housing, employment, credit, and insurance ads [10]. On August 21, 2018, Facebook announced it would eliminate 5,000 targeting options related to ethnicity or religion, such as “Native American culture”, “Passover”, “Evangelicalism”, and “Buddhism”, from being used by advertisers [68, 69].

On March 19, 2019, Facebook settled with the ACLU, the Communications Workers of America, the National Fair Housing Alliance, and others on their multiple lawsuits against Facebook over advertising discrimination [70, 71]. Facebook agreed to create a separate portal for ads for housing, employment, and credit [72]. On this portal, Facebook agreed to offer “a much more limited set of targeting options so that advertisers cannot target ads based on Facebook users’ age, gender, race, or categories that are associated with membership in protected groups, or based on zip code or a geographic area that is less than a 15-mile radius, and cannot consider users’ age, gender, or zip code when creating ‘Lookalike’ audiences for advertisers” [72].

On August 26, 2019, Facebook started implementing this agreement by requiring advertisers purchasing employment, housing, and credit ads to use “Special Ad Categories” tools instead of the regular Facebook advertising tools [2]. The “Special Ad Categories” tools restrict options that “allow targeting by age, gender, ZIP code, multicultural affinity, or any detailed options describing or appearing to relate to protected characteristics.” Advertisers must create a Special Ad Audience that finds users who are similar to their existing customers “in online behavior without considering things like age, gender or ZIP code” [2].

Before settling these lawsuits, Facebook repeatedly argued that it is not liable for discrimination on its platform that might result from the actions of its advertisers, because it is protected by Section 230 of the Communications Decency Act [26, 68]. According to Facebook, since it is a platform rather than a publisher, Section 230 means that it should not be liable for the content that it hosts [26]. In 2018, the U.S. Department of Justice filed Statements of Interest in two discrimination-related lawsuits against Facebook, arguing that Section 230 liability protections do not apply to Facebook’s advertising platform [73, 74].

Finally, in response to the July 2020 Stop Hate for Profit boycott organized by the NAACP, the Anti-Defamation League, Color of Change, and others, Facebook on July 8, 2020 released a civil rights audit conducted by Laura Murphy, former Director of the ACLU Legislative Office, and attorneys at the law firm Relman Colfax [18]. The audit focused primarily on issues of misinformation and hate speech on Facebook and how Facebook has “placed greater emphasis on free expression”rather than balancing that objective with the “value of non-discrimination”, values that the report noted do not need to be mutually exclusive [18]. For example, the audit criticized Facebook for deciding that President Trump’s post stating “when the looting starts, the shooting starts” – in regards to the May 2020 protests following George Floyd’s death – did not violate its content policies about the incitement of violence and for leaving the post up without any warning labels [18]. Regarding advertising discrimination, the audit acknowledged that HUD had filed charges against Facebook for violating fair housing laws with its ad targeting options [18]. The audit also acknowledged that research from Northeastern University and Upturn [5] had shown it is possible for the Special Ad Audiences tool to produce a biased result even though it does not use protected class information [18]. Regarding this audit and Facebook’s civil rights policies, Facebook’s Chief Operating Officer, Sheryl Sandberg, said that “it is the beginning of the journey, not the end” [19]. On August 11, 2020, Facebook announced that it would finally retire its controversial multicultural affinity advertising targeting options for racial and ethnic groups [20].

The boycott by most major advertisers ended by August 2020, and Facebook does not appear to have suffered significant financial damage as a result of the boycott. The boycott coincided with the COVID-19 pandemic, which saw many small and medium businesses increase their web presence and online sales [75]. Besides the civil rights audit, Facebook also responded to some of the demands of the boycott’s organizers, banning Holocaust denial posts and blackface posts [75] and agreeing to hire a civil rights vice president, though not a C-suite executive as demanded by the boycott’s organizers [76].

At the end of 2020, Facebook also began to update some “race-blind” policies that had a disproportionate impact on marginalized groups. In December 2020, Facebook started re-engineering its automated moderation systems, which previously did not distinguish groups that historically were targets of hate speech from groups that were not [77]. Thus, comments such as “White people are stupid” were treated the same way as anti-Semitic or racist slurs [77]. Black users complained that the old system removed posts such as “Thank a Black woman for saving our country.” Civil rights experts argued that “you can’t have the conversation if it is being filtered out, bizarrely, by overly blunt hate speech algorithms”[77]. Facebook is de-prioritizing its moderation of negative comments about “Whites”, “men”, and “Americans” as less likely to be harmful, while acknowledging that underrepresented groups need more protection [77].

This study examines changes in the discriminatory potential of Facebook’s advertising platform between January 2020 and January 2021. It was a tumultuous year, which saw organized action by advocacy groups and by Facebook’s users, and largest advertisers to apply pressure for greater civil rights protections on the platform. Facebook has also begun to publicly take some steps toward reform, such as removing multicultural affinity targeting options and re-engineering its content moderation system.

Has Facebook gone far enough to prevent discrimination on its advertising platform?

Methods

Overview

This study is the first to examine the racial and ethnic breakdowns of Facebook’s multicultural affinity groups and the similar-sounding cultural interest groups. It is also the first to examine the racial and ethnic biases of Facebook’s Lookalike Audience and Special Ad Audience tools across multiple time periods and attempt to determine whether the degree of bias is affected by populating a source audience with individuals with racially stereotypical names and ZIP codes.

I conducted the study over two waves, with Wave 1 occurring in January 2020 and Wave 2 in January 2021. During each wave, I conducted two types of tests of Facebook’s ad platform to answer my research questions about potential racial and ethnic biases.

Test 1 – Studying the Racial and Ethnic Breakdowns of Facebook Targeting Options

Detailed Targeting options
- 2020 – Multicultural Affinity groups
  - African American (US)
  - Asian American (US)
  - Hispanic (US – All)
Research question: Are the cultural interest groups as racially and ethnically homogeneous as the old multicultural affinity groups?

Test 2 – Studying the Bias in Facebook’s Lookalike Audience and Special Ad Audience Tools Using A Set Theory Approach
- Lookalike Audiences (2020 and 2021)
- Special Ad Audiences (2020 and 2021)
- Research questions:
  - Do Lookalike and Special Ad audiences reflect the racial and ethnic biases present in the lists of individuals used to generate them?
  - Is the degree of racial and ethnic bias in Lookalike and Special Ad audiences affected by well-established racial proxies from the offline world, such as the name or ZIP code of individuals used to create the audience?
  - What are the differences in the type and degree of bias observed in Facebook’s advertising tools in 2021 versus 2020?

In each wave, I used that month’s North Carolina voter list of ~8 million voters to generate source audiences to test Facebook’s advertising tools. The North Carolina voter data is useful for this study, because it includes important personal information that Facebook can use to match a voter to a user profile. It also contains self-reported race and ethnicity data that I could use to create subsets of voters for each test.

This study tested how Facebook’s ad platform relates to different samples of African-American, White, Asian, and Hispanic voters, which are racial and ethnic groups that exist in the voter data and align with Facebook targeting options. For each test, I first created lists of voters with known race and ethnic data. I then used those lists as source audiences for different Facebook ad tools to see if the estimated reach according to Facebook changed in a biased way based on the demographics of the list that was used and the Facebook tool being tested.

Preparing the North Carolina Voter Data for Testing on Facebook

The North Carolina voter dataset is publicly available and contains the following variables that overlap with what Facebook requests in order to match an uploaded list of voters with their corresponding Facebook profiles for ad targeting through the Custom Audience tool:

First name
Last name
City
State
ZIP code
Country
Gender
Age
Year of birth
Phone number

I filtered the voter data for “Active” and “Verified” voters to ensure that I used voters with the most up-to-date city and ZIP code fields to match against Facebook profiles.

Voter registration forms in North Carolina ask voters to indicate their race and ethnicity, which are captured in the state voter data (Figure 9). I used the African American/Black, Asian, and White race categories and the Hispanic and Not Hispanic/Latino ethnicity categories to create subsets of voters to test the degree of bias in Facebook’s advertising platform.

Figure 9. North Carolina Voter Registration Form from 2021. This form asks the voter to provide their demographic information in terms of ethnicity and race in the box highlighted in red.

Since Test 2 involved seeing whether the degree of bias in Lookalike and Special Ad audiences changed with the use of source lists of individuals with commonly given names of one race or ethnicity for the source audience, I started with the ethnicolr library in Python to predict a voter’s race or ethnicity based on their first and last names [78]. I then created subsets of voters of each demographic group who have commonly given names of that race or ethnicity according to ethnicolr predictions.

Research has revealed trends in the popular names that are given to babies of different races and ethnicities [37, 38, 51]. In fact, these trends are often used in political science and social science research to predict the race and ethnicity of individuals in a dataset that doesn’t contain explicit race and ethnicity fields [38]. A common approach is to use the U.S. Census Bureau’s Frequently Occurring Surnames dataset, which contains all last names occurring more than 100 times in the Decennial Census and provides a racial and ethnic breakdown in the usage of each last name [79]. However, using only last names may incorrectly predict the race of some individuals, especially African-Americans, who have last names that are also frequently used by White families. In addition, African-Americans often have more racially distinct first names. Thus, Sood and Laohaprapanon created the ethnicolr library, which uses a Long Short Term Memory neural network trained on Florida voter registration data to predict an individual’s race or ethnicity using both their first and last name [78]. They found that their model using both names, which had a precision of 83% and a recall of 84%, performed better than a model using only the last name in sample testing. Since its release in 2018, the ethnicolr library has been used in multiple research studies in medicine, political science, economics, education, and other fields [80, 81, 82, 83, 84]. Figure 10 shows the results from the ethnicolr library for four names. The model outputs probabilities that a given first and last name belongs to one of four classes: White, African-American, Asian, or Hispanic. It then predicts a label based on which class had the highest probability. The first two rows compare “Jinyan Zang” to “Jinyan Zane”, where changing the last name resulted in a shift in the predicted label from Asian to White. The last two rows compare “Latanya Sweeney” to “Tanya Sweeney.” The change in the first name resulted in the predicted label changing from African-American to White.

Figure 10. Example Results from the ethnicolr Library in Python Using Four Names. Changing “Jinyan Zang” to “Jinyan Zane” resulted in the predicted label switching from Asian toward White. Changing “Latanya Sweeney” to “Tanya Sweeney” resulted in the predicted label switching from African-American to White.

Since Test 2 also involved seeing whether the degree of bias in Lookalike and Special Ad audiences changed when source lists used voters living in racial or ethnic enclaves, I started with the U.S. Census Bureau’s American Community Survey to find the racial and ethnic breakdown of every ZIP code in North Carolina [85]. According to the Census, North Carolina’s population is 71% White, 22% African-American, 3% Asian, and i10% Hispanic [86]. I classified African-Americans living in ZIP codes with >50% African-Americans, Whites living in ZIP codes with >90% Whites, Asians living in ZIP codes with >20% Asians, and Hispanics living in ZIP codes with >20% Hispanics as having a racially or ethnically stereotypical ZIP code.

Combining ethnicolr predictions based on names with Census data identifying racial enclave ZIP codes, I created four versions of 10K voter samples for each race/ethnicity to study the demographic biases of Lookalike and Special Ad audiences in Test 2 (Figure 11).

Figure 11. 10K Voter Samples with Different Traits Used to Create Lookalike and Special Ad Audiences for Each Race/Ethnicity for Test 2 Analysis.

Test 1 – Studying the Racial and Ethnic Breakdowns of Targeting Options By Facebook

I started by recording the target size for the relevant Facebook targeting options in 2020 and 2021 using its ad planning tool, shown in Figure 12. For 2020, the options were “African American (US)”, “Asian American (US)”, and “Hispanic (US – All)”. In 2021, the options were “African-American Culture”, “Asian American Culture”, and “Hispanic American Culture”. I then compared the target size of each Facebook option to the corresponding estimate from the US Census for the 18+ population of that demographic group [87].

Figure 12. Target Size of the “African American (US)” Option in 2020. The size in January 2020 was 87,203,689 Facebook users.

For each wave, I created different Custom audiences on Facebook by segmenting the North Carolina voter list by different racial groups – African-American, Asian, and White – and by different ethnic groups – Hispanic and Non-Hispanic. I then randomly sampled 10,000 voters from each of the segments to use as the source audience to create a Lookalike audience of the 1% most similar users in the United States on Facebook.

I iterated and set each Custom or Lookalike audience as the target on Facebook’s ad planning tool (Figure 13). I then added the targeting option being tested under the “Detailed Targeting” setting of Facebook’s ad planning tool and recorded the updated daily reach estimate. In the example shown in Figure 14, an ad that targeted the African-American voters Custom audience and also matched the “African-American Culture” interest option would only reach 142,000 North Carolina users daily. This is 37% of the reach of an ad that targeted the same Custom audience without the additional “African-American Culture” criterion (Figure 13).

Figure 13. Example of the Estimated Daily Reach of the African-American Voters Custom Audience on Facebook’s Ad Planning Tool in 2021. The maximum estimated daily reach was 379K, as highlighted in the red box.

Figure 14. Example of the Estimated Daily Reach of Targeting an African-American Voters Custom Audience Who Match the “African-American Culture” Interest Option on Facebook’s Ad Planning Tool in 2021. The maximum estimated daily reach was 142K, or 37% of the reach in Figure 13.

When the estimated reach is below 1 million, Facebook’s ad planning tool in general rounds to the nearest thousand by using “K”, which may have introduced some small rounding errors into the results shown in this study. I maximized the ad budget to $1 million in most cases to ensure the maximum reach estimate is used.

Test 2 – Studying the Bias in Facebook’s Lookalike Audience and Special Ad Audience Tools Using A Set Theory Approach

While Facebook doesn’t permit direct demographic queries for a Lookalike or Special Ad audience, I was able to leverage the daily reach estimates of Facebook’s ad planning tool to indirectly observe the racial and ethnic breakdown of a given Lookalike or Special Ad audience amongst North Carolina voters by using the set theory approach described in Sapiezynski et al. [5].

For example, to study the African-American versus White bias of a Lookalike or Special Ad audience, I first created a 2 million voter sample made up of 1 million randomly sampled African-American voters and 1 million randomly sampled White voters. From the remaining voters in North Carolina not in the sample, I then created lists of 10,000 randomly sampled voters with particular racial- or ethnic-related traits to create corresponding Lookalike and Special Ad audiences of the 1% most similar Facebook users in the United States (Figure 11). Thus, to see whether a Lookalike audience based on African-Americans was biased toward including more African-American than White voters, I first measured the estimated reach of the 1 million African-American voters Custom audience (Figure 15). I then measured the estimated reach of a 1 million African-American audience that excluded the Lookalike Audience based on African-Americans (Figure 16). Finally, I repeated the process for the 1 million White voters Custom audience.

Figure 15. Example of the Estimated Daily Reach of Targeting a 1 Million African-American Voters Custom Audience in 2021. The maximum estimated reach was 299K, as highlighted in the red box.

Figure 16. Example of the Estimated Daily Reach of Targeting a 1 Million African-American Voters Custom Audience While Excluding a Lookalike Audience Based on African-American Voters in 2021. The maximum estimated reach was 160K, which is 139K fewer users than Figure 15.

In this case, the estimated reach of the 1 million African-American voters Custom audience decreased by 139,000 after excluding the Lookalike audience based on African-Americans, but the estimated reach of the 1 million White voters Custom audience decreased by only 17,000 under the same circumstances. Thus, within the intersection of the Lookalike audience and the 2 million voter sample, 89% of the overlap were African-American voters, while 11% were White voters, as shown in Figure 17. This means that African-Americans were overrepresented, being far above the 50% baseline in the 2 million voter sample. In this paper, I refer to statistics like the 89% above as the sample share of African-American voters in the Lookalike audience, and I refer to statistics like the 11% above as the sample share of White voters in the Lookalike audience. I also describe a demographic group with a size exceeding the baseline level as “overrepresented” and a group with a size below the baseline as “underrepresented”. If the set theory approach found that a Lookalike or Special Ad audience overrepresented a demographic group, I describe that audience in this paper as being “biased” toward that race or ethnicity.

Figure 17. Intersection of the Lookalike Audience Based on African-American Voters with the 1 Million African-American Voter Sample and the 1 Million White Voter Sample in 2021. Because 89%, or 139,000 users in the audience, were African-American voters and just 11% were White voters, the Lookalike audience overrepresented African-American voters and appears to be biased toward African-Americans.

There are far fewer Asian voters in North Carolina than African-American and White voters. As Figure 18 illustrates, I used a 150,000-voter sample as the comparison set, built from samples of 50,000 Asian, African-American, and White voters each, to study the degree of bias in a Lookalike or Special Ad audience toward Asians using the set theory approach. Similarly, Figure 19 illustrates the use of a 200,000-voter sample as the comparison set, half Hispanic and half Non-Hispanic, to study the degree of bias in a Lookalike or Special Ad audience toward Hispanics. Finally, since Facebook does not allow Special Ad audiences to be in the “Exclusion” position, I simply flipped the settings described earlier by first measuring the reach of the Special Ad audience on its own and then measuring the reach of the Special Ad audience while excluding a given voter sample, in order to implement the set theory approach for measuring biases in Special Ad audiences.

Figure 18. Set Theory Approach to Study the Bias Toward Asians in Lookalike or Special Ad Audiences Using a 150,000 Sample of Asian, African-American, and White Voters.

Figure 19. Set Theory Approach to Study the Bias toward Hispanics in Lookalike or Special Ad Audiences Using a 200,000 Sample of Hispanic and Non-Hispanic Voters.

Results

Test 1 – Studying the Racial and Ethnic Breakdowns of Targeting Options By Facebook

The number of Facebook users in a target group can bear little relation to US Census estimates of the population of the related ethnic or racial group. The Asian and Hispanic “culture” target groups examined in our study in 2021 were much closer to the Census estimates for Asians and Hispanics than the discontinued “affinity” groups. In 2021, Facebook’s “African-American Culture” ad targeting option contained 75% fewer White users than the old “African American (US)” option removed in the previous year. By contrast,the number of non-Asians and Non-Hispanics increased significantly for the Asian- and Hispanic-related targeting options.

In 2020, Facebook’s multicultural affinity group targeting option for “African-American (US)” included 2.49 times more people than the Census estimate of the African American adult population in the US, while “Asian American (US)” was 0.28 times the Census estimate for that ethnic group, and “Hispanic (US – All)” was 0.50 times the Census estimate for Hispanics (Figure 20). In 2021, the number of Facebook users interested in “African-American Culture” was 2.26 times the Census estimate of African-Americans, while the target size of “Asian American Culture” was 0.96 times the Census estimate of Asian-Americans and that of “Hispanic American Culture” was 1.31 times the Census estimate of Hispanic residents (Figure 20). Thus, Facebook’s Asian American Culture and Hispanic American Culture option provided a target size closer to related Census population estimates in 2021, while the target size of the African-American Culture option stayed at twice the population estimate, declining only slightly from the size of the “African American (US)” option in 2020.

Figure 20. Ratio of Facebook Advertising Target Size to Census Population Estimate. The Census population estimates are the US Census’ 2019 population estimates [87] for each race or ethnicity for individuals 18 and older, since that was the age limit for the Facebook Detailed Targeting. Facebook’s ad targeting sizes became much closer to the Census estimates by 2021 for Asians and Hispanics.

In 2020, the share of NC voters on Facebook who could be reached by targeting “African American (US)” was 43% of African-American voters, 23% of Asian voters, and 39% of White voters (Figure 21). However, since there are far more White voters than African-American voters in North Carolina, this means that the “African-American (US)” targeting option could reach approximately 150,000 African-American voters and 428,000 White voters (Figure 21). In 2021, the conditional probability of being interested in “African-American Culture” was significantly higher for African-American voters at 37%, compared to 8% for Asian and White voters (Figure 21). This resulted in 142,000 African-American voters, 109,000 White voters, and only 2,000 Asian voters being interested in “African-American Culture” (Figure 21). Thus, 75% fewer Whites were included in “African-American Culture” in 2021 than in “African American (US)” in 2020.

Figure 21. Share and Number of NC Voters on Facebook with African-American Related Targeting Attributes. 75% fewer Whites were included in the African-American Culture interest group in 2021 than in “African American (US)” in 2020.

In 2020, the share of Asian voters on Facebook reached by the “Asian American (US)” option was 8.9%, which dwarfed the 0% of African-American voters and the 0.1% of White voters (Figure 22). However, in terms of absolute numbers, this means that approximately 1,600 Asian and White voters could be reached with the same multicultural affinity option (Figure 22). In 2021, the “Asian American Culture” target option could reach more non-Asians with a 7.4% share of Asian voters, but this option also reached a 2.5% share of African-American voters and 1.4% share of White voters (Figure 22). This means that targeting “Asian American Culture” would reach only 1,400 Asian voters, far fewer than the 9,400 African-American voters and 18,000 White voters who share this interest (Figure 22).

Figure 22. Share and Number of NC Voters on Facebook with Asian-American Related Targeting Attributes. In 2021, the “Asian American Culture” target option reached far more non-Asians than Asians, with an estimated reach of 1,400 Asian voters but 9,400 African-American voters and 18,000 White voters.

In 2020, Hispanic voters were more likely than Non-Hispanic voters to be reached by Facebook’s “Hispanic (US -All)” targeting option in both relative and absolute terms. Of the Hispanic voters on Facebook in our sample, 27.3% had the “Hispanic (US – All)” attribute, compared to only 0.5% of Non-Hispanic voters, which translates into approximately 15,000 Hispanic voters and 6,400 Non-Hispanic voters (Figure 23). In 2021, only 9.8% of Hispanic voters were classified as interested in “Hispanic American Culture”, and 1.1% of Non-Hispanic voters had the same interest (Figure 23). This results in only 5,800 Hispanic voters but 16,000 Non-Hispanic voters being reached by the targeting option, since there are far more Non-Hispanic than Hispanic voters on Facebook (Figure 23).

Figure 23. Share and Number of NC Voters on Facebook with Hispanic-American Related Targeting Attributes. In 2020, more than twice as many Hispanic voters as Non-Hispanic voters are targeted by “Hispanic (US – All)”, but that ratio flips in 2021, with more than twice as many Non-Hispanics as Hispanics reached by the “Hispanic American Culture” targeting option.

I also found that in 2021, Lookalike audiences based on African-American, Asian, White, or Hispanic voters tended to have shares interested in “African-American Culture” or “Hispanic American Culture” that were similar to those ethnic group’s shares of the corresponding voter lists. However, all Lookalike audiences, regardless of which voter list was used to create them, had about 2-3% of their users interested in “Asian American Culture”, including the Lookalike audience based on Asian voters. For more details about these results, see Appendix A.

Test 2 - Studying the Bias in Facebook’s Lookalike Audience and Special Ad Audience Tools Using A Set Theory Approach

In both 2020 and 2021, Facebook’s Lookalike and Special Ad audiences overrepresented the sample share of either African-American or White voters, depending on which race was dominant in the customer list used as the source audience. This bias increased when using customer lists with stereotypically African-American or White names or ZIP codes. Similar biases were observed for Lookalike audiences based on Asians or Hispanics. In one case, a customer list of Asians with stereotypical Asian names and ZIP codes generated a Lookalike audience having a 100% sample share of Asian voters. In a shift from 2020, I found that Special Ad audiences based on Asians did not significantly overrepresent Asian voters in 2021.

When studying the breakdown of the 2 million NC voter sample intersecting with Lookalike audiences, 83% of the overlap between the sample and a Lookalike audience based on African-Americans in 2020 were African-American voters. That overlap increased to 89% in 2021 (Figure 24). As the customer list used to create the Lookalike audience took on more stereotypically African-American traits by being limited to voters with commonly given African-American names, voters who live in a ZIP code with >50% African-Americans, or both, the sample shares of African-American voters in the Lookalike audiences increased to 93% in 2020 and to 94% in 2021 (Figure 24).

Figure 24. Sample Share of African-American Voters in Lookalike Audiences Based on Lists of NC Voters with Different Traits. As the customer list used to create the Lookalike audience contained more stereotypically African-American traits in terms of names and ZIP codes, the sample shares of African-American voters in the corresponding Lookalike audiences increased, reaching 93% in 2020 and 94% in 2021.

Similarly, I also found that Lookalike audiences based on Whites contained a large majority of White voters in their overlap with the 2 million NC voter sample in both waves. In 2020, 73% of the voters shared between the 2 million voter sample and the Lookalike audience based on White voters were White. In 2021 the rate was similar at 71% (Figure 25). As the White voters used to create the Lookalike audience took on additional White-affiliated traits by having commonly given White names, living in a >90% White ZIP code, or both, the sample shares of Whites peaked at 87% in 2020 and 84% in 2021 for the resulting Lookalike audiences (Figure 25).

Figure 25. Sample Share of White Voters in Lookalike Audiences Based on Lists of NC Voters with Different Traits. Lookalike audiences based on Whites overrepresented White voters in both waves, with a sample share of 73% White voters in 2020 and 71% in 2021.

In 2020, Lookalike audiences based on Asian voters appeared to have a moderate tendency to favor Asians. This became a significantly stronger trend in 2021. In 2020, in a comparison of the 150,000-voter sample with Lookalike audiences based on Asian voters, 51% of the voters who overlapped were Asian. This overlap increased to 83% for the Lookalike audience based on Asian voters who had commonly given Asian names and lived in Asian enclave ZIP codes (Figure 26). In 2021, the overlap rates started at 68% for the Lookalike audience based on Asians and increased to 100% for the Lookalike audience based on Asians with stereotypically Asian traits of name and ZIP code (Figure 26). In all cases, the sample shares of Asian voters were far above the 33% baseline in the 150,000-voter sample. It is possible that the actual sample share of Asian voters in the most extreme case was slightly below 100%. A few Black and White voters could have been part of the intersection between the Lookalike audience and the voter sample, but those voters might not have been captured because Facebook rounds the advertising reach estimate on its ad planning tool to the nearest 1,000.

Figure 26. Sample Share of Asian Voters in Lookalike Audiences Based on Lists of NC Voters with Different Traits. In 2021, Lookalike audiences based on Asians were significantly biased toward including Asian voters, reaching a 100% sample share of Asian voters in one case.

In both 2020 and 2021, Lookalike audiences based on Hispanics were biased toward including more Hispanic voters. In 2020, Hispanic voters accounted for 71% of the overlap between the 200,000-voter sample and the Lookalike audience based on Hispanics. A similar 69% share was seen in 2021 (Figure 27). As the customer list used to create the Lookalike audience appeared more stereotypically Hispanic by name or ZIP code, the sample share of Hispanic voters increased slightly to 75% in 2020, and 79% in 2021 (Figure 27).

Figure 27. Sample Share of Hispanic Voters in Lookalike Audiences Based on Lists of NC Voters with Different Traits. In both 2020 and 2021, Lookalike audiences based on Hispanics included more Hispanic voters than Non-Hispanic voters, with a sample share of 71% Hispanic voters in 2020 and 69% in 2021.

Even though Facebook created Special Ad audiences as an anti-discrimination tool for housing, employment, and credit-related ads, I found that Special Ad audiences could still be biased toward including more African-American or White voters, depending on the demographics of the source audience.

In 2020 and 2021, the Special Ad audiences based on African-Americans demonstrated significant biases toward including more African-American voters. In 2020, the sample shares of African-American voters in the relevant Special Ad audiences started at 83% and went up to 97% for the Special Ad audience based on African-Americans with stereotypically African-American names and ZIP codes (Figure 28). These sample shares are similar to those observed for the Lookalike audiences based on African-Americans shown in Figure 24. In 2021, slightly fewer African-Americans were in the intersection of the 2 million voter sample with the relevant Special Ad audiences, starting at a 76% sample share and increasing to 89% for the Special Ad audience based on a list of African-American voters with commonly given African-American names living in ZIP codes with >50% African-Americans (Figure 28).

Figure 28. Sample Share of African-American Voters in Special Ad Audiences Based on Lists of NC Voters with Different Traits. In both waves, the Special Ad audiences based on African-Americans demonstrated significant biases toward including more African-American voters, with a sample share of 83% African-American voters in 2020 and 76% in 2021.

Special Ad audiences based on Whites also exhibited strong biases toward including more White voters, with sample shares similar to the Lookalike audiences shown in Figure 25. In 2020, White voters accounted for 83% of the overlap between the 2 million voter sample and the Special Ad audience based on White. In 2021, the sample share was 81% (Figure 29). In 2020, the Special Ad audience based on Whites with commonly given White names had the largest bias, with a 90% sample share of White voters (Figure 29). In 2021, the Special Ad audience based on Whites living in >90% White ZIP codes had the largest bias, with a sample share of 91% of White voters (Figure 29).

Figure 29. Sample Share of White Voters in Special Ad Audiences Based on Lists of NC Voters with Different Traits. In 2020, White voters were overrepresented, constituting 83% of the overlap between the 2 million voter sample and the Special Ad audience based on Whites. The same was true in 2021, with the sample share of White voters being 81%.

Interestingly, while in 2020 Special Ad audiences based on Asian voters demonstrated a bias toward Asians, that bias was significantly reduced in 2021. For example, in 2020, the sample share of Asian voters in Special Ad audiences based on different types of Asian voters started at 44% and increased to 67% when Asians with stereotypically Asian names and ZIP codes were used to create the Special Ad audience (Figure 30). In 2021, those sample shares go from 36% to 44% for the corresponding Special Ad audiences (Figure 30). Considering that the 150,000-voter sample used for each test has a baseline of 33% or 50,000 Asian voters, this means that Asian voters were only slightly overrepresented in the intersection of the voter sample and the relevant Special Ad audiences in 2021. This contrasts significantly with the very strong bias toward Asians, up to 100% sample share in one case, observed in Lookalike audiences based on similar customer lists of Asians shown in Figure 26.

Figure 30. Sample Share of Asian Voters in Special Ad Audiences Based on Lists of NC Voters with Different Traits. Interestingly, while in 2020 Special Ad audiences based on Asian voters demonstrated a bias toward Asians, that bias was significantly reduced in 2021, with sample shares of Asian voters within 8 percentage points of the expected baseline of 33% for all four tests of Special Ad audiences.

I found that Special Ad audiences based on Hispanic source audiences with varying attributes were not consistently biased toward including more Hispanic voters across the two waves. In 2020, 3 of the 4 Special Ad audiences based on Hispanics had a sample share of Hispanic voters more than 10 percentage points above the 50% baseline in the 200,000-voter sample (Figure 31). In 2021, bias was also evident in 3 of the 4 Special Ad audiences, though a slightly different combination of audiences (Figure 31). For example, the Special Ad audience based on Hispanics with commonly given Hispanic names had a relatively low 47% sample share of Hispanic voters in 2020, while a Special Ad audience based on similar individuals in 2021 had a much higher 82% sample share of Hispanic voters (Figure 31). These patterns contrast with the consistent overrepresentation of Hispanic voters in Lookalike audiences based on Hispanics shown in Figure 27.

Figure 31. Sample Share of Hispanic Voters in Special Ad Audiences Based on Lists of NC Voters with Different Traits. Special Ad audiences based on different types of Hispanics did not consistently exhibit bias toward including more Hispanic voters in 2020 and 2021.

Discussion

This study shows that despite the advertising boycott and Facebook’s response to pressure on civil rights issues in 2020, there remained multiple ways to use the tools of Facebook’s advertising platform to discriminate by race and ethnicity in 2021. These include targeting options, Lookalike Audiences, and Special Ad Audiences.

Key Findings

The results of this study support the following key findings:

In 2021, Facebook’s “African-American Culture” ad targeting option provided audiences with 75% fewer White users than the old “African American (US)” option removed in the previous year. On the other hand, targeting options related to Asians and Hispanics included more non-Asians and Non-Hispanics in 2021. In 2020, 43% of African-American voters and 39% of White voters can be reached by the “African American (US)” targeting option. In 2021, those rates were 37% and 8% respectively for the “African-American Culture” targeting option (Figure 21). In 2021, the number of White voters reached by the “Asian American Culture” targeting option increased by 16,400 compared to the White voters reached by the “Asian American (US)” option in 2020, while the number of Asian voters reached by the two options declined by 200 over the same period (Figure 22). Similarly, the number of Hispanics reached by the “Hispanic American Culture” targeting option in 2021 was 9,200 fewer than the reach of the “Hispanic (US – All)” option in 2020, while the number of Non-Hispanics increased by 9,600 over the same year (Figure 23).
Facebook’s tools to help advertisers find similar users to their existing customers allow advertisers to create audiences that include more African-Americans or Whites depending on which racial group is dominant in an advertiser’s customer list. In my tests, this was true for both the Lookalike Audience tool and the Special Ad Audience tool that Facebook designed to explicitly not use sensitive demographic attributes when finding similar users. In 2020, the sample share of African-American voters in Lookalike audiences based on a source audience of African-Americans was 83%. This increased to 89% in 2021 (Figure 24). For Special Ad audiences based on the same list of African-Americans, in 2020, the sample share of African-American voters was 83%, and in 2021, it was 76% (Figure 28). Lookalike audiences based on Whites saw a slightly smaller degree of bias with a 73% sample share of White voters in 2020 and a 71% sample share in 2021 (Figure 25). Special Ad audiences based on White voters had a 83% sample share of White voters in 2020 and 81% in 2021 (Figure 29).
The degree of bias toward including more African-Americans or Whites in a Lookalike or Special Ad audience was larger when customer lists of individuals with racially stereotypical names or ZIP codes were supplied as the basis for each tool. Lookalike audiences using a list of African-Americans with commonly given African-American names or living in ZIP codes with >50% African-Americans had sample shares of African-American voters up to 93% in 2020 and up to 94% in 2021 (Figure 24). Special Ad audiences based on similar customer lists had up to 97% sample shares of African-American voters in 2020 and 89% in 2021 (Figure 28). Lookalike audiences based on Whites with stereotypically White names or ZIP codes had sample shares of White voters up to 87% in 2020 and 84% in 2021 (Figure 25).
Similarly, Lookalike audiences can also become biased toward Asians, reaching up to 100% Asian in one case using a customer list of Asians with stereotypical names and ZIP codes. Lookalike audiences based on Hispanics overrepresented Hispanics. In a shift from 2020, Special Ad audiences based on Asians in 2021 did not appear to overrepresent them. Figure 26 shows that the sample share of Asian voters in a Lookalike audience based on Asians is 51% in 2020 and 68% in 2021, compared to a baseline of 33% Asian in the 150,000-voter sample. The sample share of Asian voters increased to 83% in 2020 and 100% in 2021 for the Lookalike audience based on Asians with commonly given Asian names and living in ZIP codes with >20% Asians. Special Ad audiences based on Asians in 2020 had a 44% sample share of Asian voters. The share increased to 67% for audiences based on Asians with stereotypically Asian names and ZIP codes (Figure 30). On the other hand, in 2021, all of the Special Ad audiences based on Asians had sample shares of Asian voters within 8 percentage points of the expected baseline of 33%. Finally, in 2020, the sample shares of Hispanic voters in Lookalike audiences based on different types of Hispanics ranged from 70-75% and in 2021 ranged from 69% to79%, well above the 50% baseline share of Hispanics in the 200,000 voter sample (Figure 27).

I found that removing “African-American (US)”, “Asian American (US)”, and “Hispanic (US – All)” as targeting options in August 2020 did not mean that the similar-sounding cultural interest groups available in 2021 could not be used for racial and ethnic targeting in 2021. In fact, 75% fewer Whites were targeted by the “African-American Culture” option in 2021, compared to the “African-American (US)” option in 2020. I also found that Lookalike and Special Ad audiences can become biased to include either more African-Americans or more Whites, based on which race is more dominant within the customer list used as the source audience, especially if the customers also have racially stereotypical names and ZIP codes. The sample shares of African-American voters reached up to 93%-94% for some Lookalike audiences in 2020 and 2021. Lookalike audiences based on Asians or Hispanics were also biased toward overrepresenting the corresponding demographic group in both 2020 and 2021, with the degree of bias reaching up to a 100% sample share of Asian voters for a Lookalike audience based on Asians with stereotypical Asian names and ZIP codes. One shift in 2021 was that Special Ad audiences based on Asians did not appear to overrepresent them, a change from 2020.

Limitations of This Study

While the results of this study describe how Facebook’s ad targeting algorithms could bias the race or ethnicity of advertising audiences in 2020 and 2021, they are not able to fully describe why these biases occur. Even if an algorithm wasn’t designed to be discriminatory, bias can occur through a series of different mechanisms.

Underlying biases in the decisions made by humans may be reflected in the training data [60].
The training data may be unrepresentative or incomplete [52, 88, 89].
A reinforcement learning algorithm may become biased over time due to the biased behaviors of users [57, 90].
Trade-offs in balancing an algorithm’s performance in different fairness and accuracy metrics may result in biased outcomes for different demographics groups [65, 91, 92, 93, 94, 95, 96, 97, 98].
A complex algorithmic decision-making system like Facebook’s ad platform does not have just one algorithm but rather a series of different algorithms interacting with one another, so even if individual algorithms are not biased on their own, their interactions may result in biased outcomes [99].

In this case, it is possible that a combination of multiple causes contributed to the biased outcomes observed in this study. Researchers have found that racial and ethnic groups tend to behave differently from each other online. They visit different websites [100, 101, 102, 103], follow different social media [104, 105], and even browse the web using different devices [106, 107]. In addition, as an online social network, Facebook also has data on the friends of each user. Researchers have found that Americans tend to have very racially homogeneous friend networks [108]. For White Americans, on average 91% of the members of their social network are also White [108]. For Black Americans, on average 83% of their social network are also Black [108]. Similarly, 75% of White Americans and 65% of Black Americans report having a core social network, defined as “people with whom they discuss important matters,” entirely of their own race [108]. Future work would need to be done, ideally with deeper access to Facebook’s data and systems than possible with this external digital audit approach, in order to explain why Facebook’s advertising algorithms are biased.

Finally, this study focused on how Facebook’s algorithms treated African-Americans, Whites, and Asians as racial groups and Hispanics and Non-Hispanics as ethnic groups. This focus was chosen because these demographic groups are the focus of Facebook’s own ad targeting options and also exist as categories in the North Carolina voter data used for testing. Future studies may also examine other demographic groups such as Native Americans, multiracial individuals, and others. For studies of smaller demographic groups, the way that Facebook generally rounds the reach estimate on its ad planning tool to the nearest thousand may require other approaches than the set theory approach used here, since the reach estimate may not change by more than 1,000 users when using different ad targeting settings.

Implications for Facebook

While not every case of advertising discrimination is illegal or even potentially undesirable, this study highlights how the lack of transparency by Facebook to the public and to its advertisers about how its ad platform can potentially discriminate by race and ethnicity may be exploited by discriminatory advertisers while undermining the goals of non-discriminatory ones. For example, discriminatory advertisers may already know that the “African-American Culture” targeting option contains fewer White users than the “African-American (US)” option Facebook removed in 2020. Discriminatory advertisers may be using proxy variable techniques similar to the ones tested in this study based on racially stereotypical names and ZIP codes to create biased Lookalike and Special Ad audiences. On the other hand, non-discriminatory advertisers may be unintentionally choosing similar targeting settings while unaware of how Facebook’s ad platform is carrying out racially and ethnically biased targeting on their behalf.

Investigations of Facebook by regulators have uncovered evidence that the tools of Facebook’s ad platform examined in this study are being used to target political as well as housing, employment, and credit ads by race and ethnicity. For example, when Congress investigated Russian interference in the 2016 elections through tactics that included the purchase of political ads on Facebook, they discovered that Russian-affiliated advertisers used multicultural affinity groups as targeting options, including the “African-American (US)” option on Facebook [15]. In a different investigation of discrimination in housing, employment, and credit ads on Facebook by the Washington State’s Attorney General’s Office, which occurred over 20 months from 2016 to 2018, investigators found that “some third-party advertisers have chosen to target advertisements they created and placed on the Facebook platform using ethnic affinity targeting options” [27, 117].

Except when required by these regulatory investigations, Facebook has not disclosed relevant ad targeting information in its publicly accessible Ad Library that would be needed to detect racial discrimination by an advertiser. As of January 2021, Facebook’s Ad Library publishes limited data about political, housing, employment and credit-related ads [109]. For political ads, Facebook publishes metadata about how much was spent, how many viewed the ad, and who saw the ad by gender and state (Appendix C). Facebook does not release data in the Ad Library about which targeting options, such as the racially affiliated interest groups studied here, were used. It also does not release information about the racial and ethnic breakdown of who saw an ad. For housing, employment, and credit-related ads, no metadata about who was targeted or who saw the ad is released (Appendix C). This is especially problematic in light of the final version of the U.S. Department of Housing and Urban Development’s (HUD) “Implementation of the Fair Housing Act’s Disparate Impact Standard” published on September 24, 2020. This regulation requires a plaintiff to present evidence of a “robust causal link” in order to bring a disparate impact discrimination lawsuit in the first place [110]. This study demonstrates how Facebook’s targeting options—Lookalike Audiences, and Special Ad Audiences—can be used to discriminate by race or ethnicity, but Facebook’s Ad Library doesn’t currently release any data that could document a “robust causal link” between how an advertiser is using Facebook’s tools and the discriminatory impact on who sees their ads.

Currently, Facebook’s anti-discrimination efforts are concentrated on limiting the targeting options for “Special Ads” related to housing, employment, or credit. Facebook disables the use of such sensitive targeting options for Special Ads as multicultural affinity groups in 2020 and cultural interest groups in 2021 (Appendix C). Facebook only allows an advertiser to add more attributes as Detailed Targeting options and does not allow an advertiser to exclude any attribute (Appendix C). Facebook also provides notices on its ad planning tool to encourage advertisers to not discriminate when targeting a Special Ad (Appendix C). Finally, Facebook does not allow Special Ads to use regular Lookalike audiences. Rather, they have to target Special Ad audiences, which Facebook designed to not use sensitive demographic attributes such as “age, gender or ZIP code” in considering which users to include [2]. However, this study found that in 2020 and 2021 Special Ad audiences could become racially biased at rates similar to Lookalike audiences when an advertiser used the same demographically homogeneous customer list to create both types of audiences. Regular ads not related to housing, employment, or credit do not face any of these restrictions or notices.

In order to better tackle discrimination in the future, Facebook can leverage its data and analytical capabilities to better detect potential racial and ethnic discrimination for both “Special” and regular ads. Right now, Facebook’s ad planning tool provides daily reach estimates given any combination of different ad targeting options, Custom audiences, Lookalike audiences, or Special Ad audiences, which is how I collected data about Facebook’s ad platform for this study. As a potential feature, Facebook can enrich its estimated reach report by displaying who will see an ad on the basis of race, ethnicity, gender, age, geography, and other categories. If a particular Custom, Lookalike, or Special Ad audience is racially or ethnically biased, Facebook can flag those audiences when they are first created in order to notify the advertiser and potentially limit their usage.

In order to carry out these digital audits for potential racial and ethnic biases in an advertiser’s target audiences, Facebook has a number of ways to collect or infer racial and ethnic data about its users. One option is similar to the North Carolina voter registration form, which asks a user to voluntarily provide their race and ethnicity. Facebook currently requests gender and date of birth on its account sign up page and includes an option for a “Custom” gender where a user can select their preferred pronouns and textbox for a preferred gender label. Facebook could adapt this approach to collect racial and ethnic data directly from its users. It is possible that many would find it unappealing to give Facebook more data about themselves, given past controversies over how Facebook has handled user data, such as the Cambridge Analytica scandal [111]. Another option is for Facebook to infer the data indirectly by following other examples in the tech industry such as Airbnb’s Project Lighthouse, which was launched in 2020 to study the racial experience gap for guests and hosts on Airbnb [112]. Project Lighthouse used a third-party contractor to assess the perceived race of an individual based on their profile picture and name [112]. Another approach is to use the name alone to infer race and ethnicity by using algorithmic approaches such as the ethnicolr Python library [78] or the U.S. Census Bureau’s Frequently Occurring Surnames dataset [79]. In fact, there is precedent at Facebook for doing exactly this type of analysis. In December 2009, Facebook researchers Lars Backstrom, Jonathan Chang, Cameron Marlow, and Itamar Rosenn published a paper on the race and ethnicity of Facebook users from January 2006 to January 2009 by comparing the last names of users to the U.S. Census’s Frequently Occurring Surnames dataset [113]. They found that Facebook was becoming increasingly diverse over time by having more African-American and Hispanic users, a result cited in a The Wall Street Journal article titled “Facebook Touts Diversity of Its Members” [114].

How “Fairness Through Unawareness” Doesn’t Prevent Algorithmic Discrimination

Finally, this study has ramifications beyond Facebook in terms of how to detect and address the issue of algorithmic discrimination in an increasingly digital world. Many of the anti-discrimination changes that Facebook implemented in recent years to its advertising platform are examples of trying to achieve “fairness through unawareness” [63], the idea that discrimination is prevented by eliminating the use of protected class variables or close proxies. For example, Facebook explicitly created the Special Ad Audiences tool – as an alternative to Lookalike Audiences – to block the use of sensitive attributes such as “age, gender or ZIP code” in considering which users are similar enough to the source audience to be included [2]. However, this study demonstrates that Special Ad audiences based on African-Americans or Whites can be biased toward the race that is more dominant in the customer list used to create the audience, just like the corresponding Lookalike audiences. In fact, even though Facebook designed its Special Ad Audience tool to explicitly not use ZIP codes as part of its algorithm, in 2021, the sample shares of African-American voters were 12 percentage points higher for Special Ad audiences based on African-Americans with stereotypically African-American ZIP codes versus African-Americans from anywhere in North Carolina (Figure 28). Likewise, in 2021, the sample shares of White voters were 10 percentage points higher for Special Ad audiences based on Whites from >90% White ZIP codes versus Whites from anywhere in North Carolina (Figure 29).

Statistics research labels this phenomenon as the Rashomon effect or the multiplicity effect [115]. This means that given a large dataset with many variables, there exists a large number of potential models that can perform approximately equally as well as a prohibited model that uses protected class variables [63]. Thus, even though the Special Ad Audiences algorithm for finding similar users to a customer list does not use demographic attributes in the same way as the Lookalike Audiences algorithm, the two algorithms may end up making functionally comparable decisions on which users are considered similar enough to be included.

In recent years, there have been regulatory efforts to promote “fairness through unawareness” as a means of protecting companies from the liability of a discrimination lawsuit. This study illustrates that Facebook’s “fairness through unawareness” changes, such as its Special Ad Audiences tool, do not necessarily prevent discrimination on the platform, though the tool may have removed the ability for plaintiffs to successfully sue Facebook for discrimination if the proposed federal policies were implemented. The initial language of the “Implementation of the Fair Housing Act’s Disparate Impact Standard” rule by the U.S. Department of Housing and Urban Development’s (HUD), which was drafted in response to the Supreme Court’s Texas v. Inclusive Communities decision and released on August 19, 2019, stated that a defendant may successfully argue its model is not discriminatory if the model does not rely on “factors that are substitutes or close proxies for protected classes under the Fair Housing Act” [116]. After public comments criticized this language, it was removed in the final rule published on September 24, 2020 [110]. This study found that Facebook’s Special Ad Audience tool can enable racial and ethnic biases in ad audiences even though it does not rely on sensitive attributes that are likely related to protected classes, which is the defense criteria established in the initial language of HUD’s disparate impact rule.