Research: How Companies Committed to Diverse Hiring Still Fail

We are in the midst of a much overdue national conversation around systemic racial bias and inequality. This reckoning has forced companies, large and small, to contend with their own diversity, and many firms have identified that they are lacking.

Firms can address these issues as the economy continues to rebound and workers reshuffle across industries to meet new labor demands. To capitalize on this opportunity, however, firms — even firms with active diversity and inclusion initiatives — must examine whether parts of their hiring processes have unintentional bias built in.

A large body of academic research has found that hiring managers display bias against underrepresented minorities. A prominent example is a 2004 study of racial bias in which researchers used a “resume audit.” They sent the same resumes out to many employers with a randomly assigned name at the top. They found that a resume generated fewer interview requests when it had a name chosen by the researchers to suggest the candidate was Black (such as “Lakisha” or “Jamal”) than when it had a name that was chosen to sound white (such as “Emily” or “Greg”).

These findings have been replicated many times, but the “resume audit” method can only investigate bias among employers willing to interview unknown candidates who apply through job ads. These employers are typically small and recruiting for less-selective positions.

We wanted to explore the preferences of larger, more prestigious employers — the type that hire through relationships with schools and other feeder organizations rather than cold resumes. Many of these employers have active diversity and inclusion initiatives, suggesting they might favor diverse candidates in their hiring, or at least avoid being biased towards minorities and women.

To investigate whether or not this was the case, we needed a new tool. So, we built one. Along with our former graduate student, Colin Sullivan, we developed what we call “incentivized resume rating.” Our method invites firms to rate resumes with randomly assigned characteristics (e.g., GPA, major, work experience, extracurricular activities, and skills) as well as a name that is indicative of race and gender. Recruiters know the resumes are hypothetical, but with an important twist: Their responses are used to match them with real job seekers, providing an incentive for firms to carefully rate the hypothetical resumes.

We first implemented incentivized resume rating with employers engaged in on-campus recruiting for graduating seniors at the University of Pennsylvania, our home institution. Hiring managers from prestigious firms signed up to participate in our program, rating 40 resumes each, and giving us a fascinating window into their hiring preferences. The results of our study were recently published in the American Economic Review.

What Did We Find?

We asked firms to rate, on a 1–10 scale, how interested they would be in hiring a potential candidate. As expected, firms valued higher GPAs — and so we benchmarked the size of our other effects in GPA points — and they preferred candidates enrolled in prestigious schools within Penn (e.g., the Wharton School of Business) more than those in the standard undergraduate program.

Also as expected, firms valued the quality and quantity of internship experience:

• Having held a prestigious internship (think Goldman Sachs, McKinsey, and Google) rather than a more standard internship in the summer before senior year increased ratings by about the same as half a GPA point (for example, going from a 3.5 to a 4.0).
• Having an additional standard internship in the summer before junior year was also valued, equivalent to about 0.25 of a GPA point.

In contrast, however, having a summer job before junior year — the kind of job a student might take to earn money (think barista, server, or lifeguard) — did not increase employer ratings at all.

This result suggests our firms did not value the skills developed on these jobs. It also introduces a socioeconomic bias: The standard internships that firms valued were typically unpaid, and some candidates could not afford to take them and had to work for money instead.

This was a first hint that how firms were making their decisions may not always serve their goal of increasing diversity. We next turned to look at the role of race and gender in resume ratings.

The recruiters in our study told us that they were seeking diverse candidates. When we surveyed the firms, 90% reported that increasing gender and racial diversity was a factor they considered positively in their hiring. Their resume ratings suggested otherwise:

• We found no aggregate preferences for minority or female candidates.
• We found that firms hiring in STEM fields rated minority and female candidates significantly lower than white males. Among STEM majors, to get the same rating as a white male with a 3.75 GPA, a minority or female candidate needed a 4.0.

In addition, our data showed that female and minority candidates received less credit for the prestigious internships described above. A prestigious internship on a white male resume boosted their ratings by 50% more than that same internship on a female or minority resume.

This result aligned with findings from the 2004 resume audit, which found that increasing resume quality helped candidates with white names substantially more than those with black names.

We also found evidence of a new type of discrimination.

In addition to asking firms how interested they would be in hiring each candidate, we also asked them how likely each candidate would be to accept a job at their firm. We saw this question as an important indicator because employers are unlikely to invest time and energy interviewing and recruiting candidates who are unlikely to join their organization. Across all fields, employers rated minority and female candidates less likely to accept a job if offered.

This result suggested that firms might be less likely to pursue minority and female candidates, fearing that they would be too hard to hire.

So, why would firms appreciate the same experience less when it was held by a minority or a woman? And why might they believe a minority or female candidate would be less likely to accept a job?

Our speculation is that employers believe that other firms in their industry are favoring diverse candidates in hiring. They might therefore believe that minority or female candidates have an easier time getting prestigious internships (making those internship a less compelling signal of candidate quality for minority or female applicants). Similarly, they might also believe that minority and female candidates will be harder to recruit because many other firms will be going after them.

But our data suggests that these beliefs are wrong. We find no evidence of firms displaying positive preferences for diversity. Indeed, firms hiring in STEM fields displayed a bias against minorities and women.

To reconcile these results, we looked at whether firms might have unconscious bias against minority and female candidates. Such unconscious bias might be “undoing” the stated positive preferences for diversity when rating resumes. We did find evidence of unconscious motives: The bias against minorities and women got larger as raters became more fatigued and spent less time evaluating each resume.

Next Steps for Organizations

Our research suggests that firms may have unconscious biases that pull the wrong candidates into their recruiting funnel, in a variety of ways. How can firms address this?

The first step requires taking a hard look at your hiring practices. This step might involve quantifying the fraction of applicants who have historically come from different demographic backgrounds and looking at the fraction of each group that gets invited to an interview and is offered a job.

If the data suggests a problem, the next step is diagnosing it. Importantly, don’t assume that well-intentioned individuals will necessarily produce neutral hiring screens. Our research revealed that there may be a gap between a firm’s diversity objectives and their actual decision-making on the ground.

Based on that assessment, the next step is to develop a plan to address these issues. Be mindful that some quick fixes introduce problems of their own. Machine-learning algorithms that are trained on historical data might “bake in” historical bias, for example. And approaches that redact candidate names do not necessarily remove all information that identifies a candidate’s race or gender, which can allow unconscious bias to creep in unchecked.

Research shows that slower, more deliberate assessments, based on objective criteria, are less prone to bias. One approach is to ask resume screeners to slow down and to use objective criteria in comparing resumes to one another. For some firms, this may not be practical, and a “quick screen” may still be required. A low-tech solution might be both redact names and any information that could signal a candidate’s race or gender. When we built our tool, we used real resume components but made them race and gender neutral (for example, stripping gendered language from clubs, activities, and sports teams).

More sophisticated approaches are also possible. In our Incentivized Resume Rating approach, the employer’s hiring preferences captured by the tool (GPA, work experience, and leadership) are used to screen real resumes and create matches, except data on gender and race is ignored. This neutralizes one of the problems with most algorithmic approaches. Thus, the firms who participated in our study were matched with a more diverse set of candidates than they might have screened on their own.

Ultimately, our research allows a peek under the hood of big prestigious firms, where we found a surprising amount of race and gender bias given that these firms claim to be seeking diversity. To answer the call of the current moment, firms need to take a hard look at their hiring processes and face up the fact that they may not be as diversity-loving in practice as they are in intention.