Using lotteries to mitigate bias in hiring and promotions
To reduce the negative effects of bias, we must rethink the fundamentals of our processes.
The usual process
When it’s time to choose a new team lead (for example), a manager might decide which highly skilled[1] team members could do the job well, based on their merits[2] and potential[3]. The manager might also ask other managers for referrals[4]. Then, the manager might offer the role to the candidate that seems the best fit[5] and who seems to want to take that next career step[6].
This sounds reasonable. And it happens every day in companies and institutions around the world. But here’s how bias creeps into that process:
[1] Perception of ability
Bias affects the way someone’s competence is perceived. For example, women are held to a higher standard, must work harder to prove their competence, and are respected less than men. Women who present themselves as competent incur a penalty from having violated the caring vs competent gender norm.
We tend to assume that a man knows what he’s talking about until he proves otherwise. Whereas for women it’s all too often, the other way round, and as a result, women tend to be underestimated more. They tend to be interrupted, more talked over more. They have to prove their competence more and we often feel uncomfortable when they’re in positions of authority.
— The Authority Gap: Why Women Are Still Taken Less Seriously Than Men
[2] Merit
When it comes to considering someone’s merits, that usually means looking at past performance. The problem is that bias affects who is offered high-profile, highly visible projects. For example, women are often expected to pick up so-called non-promotable work. This begins right at the start of people’s career, sometimes called the broken rung problem or the sticky floor problem. And over time, the gap in accomplishments between people offered more opportunities and those who are not is compounded, creating what is sometimes called the thin file problem.
For every 100 men who are promoted from entry-level roles to manager positions, only 87 women are promoted, and only 82 women of color are promoted. As a result, … women can never catch up … the gender gap for women in technical roles is more pronounced, with only 52 women being promoted to manager for every 100 men.
— Women in the workplace
— Repairing the broken rung on the career ladder for women in technical roles
[3] Potential
Trying to assess someone’s potential, guessing what they might be able to do in the future, is fraught with bias. Managers underestimate womens’ potential and women get lower ratings for potential than men.
Women are often hired and promoted based on past accomplishments, while men may be hired and promoted based on future potential
— Women in the workplace
[4] Fit
The idea of someone “being a good fit” is vague and open to many biases. For example, “fit” might mean likeable. But women who present themselves as competent or as leaders are often seen as unlikeable. “Fit” might boil down to a person having attended a favored private school or being a member of a certain social club, which is biased against people of a different social class, ethnicity, and wealth. Characteristics that are often described as “professionalism”, such as hair style, dress, speaking style, and attitudes to work culture, are often premised on whiteness and Westernness as superior or as the default. “Fit” might also mean “agrees with me”, which can be counterproductive.
Too often when they say ‘culture fit’, people with hiring power really mean ‘who I personally like’ and ‘who reminds me of me’
— Why ‘Culture Fit’ Is A Failed Idea In American Hiring
[5] Ambition
Wanting to advance your career seems like a prerequisite for being considered for a new role or promotion. But when women put themselves forward, act assertively, authoritatively, they are perceived negatively. That’s because those behaviors are seen as unfeminine. So women who are trying to “do all the right things” to get ahead in their career end up experiencing a backlash for violating female gender stereotypes.
When women did all the things they have been told will help them get ahead — using the same tactics as men — they still advanced less than their male counterparts and had slower pay growth.
— The Myth of the Ideal Worker: Does Doing All the Right Things Really Get Women Ahead?
[6] Referrals
If evaluating someone’s ability, merit, and potential is already difficult, referrals are the second-hand version of that, complete with all the biases listed above. In addition, these informal methods of career advancement are often driven through male-dominated social networks which are difficult for women to access. As a result of all of this, female workers are referred less often.
Highly skilled women are systematically disadvantaged by referral-based hiring — Do Job Networks Disadvantage Women?
Designing fairness into processes
Learning about bias and fairness and becoming aware of our own thought processes is hugely important. But when it comes to achieving concrete change for the better, anti-bias training has its champions and its detractors. In other words, it might not be possible for decision makers to just not be biased through willpower alone.
Another approach for dealing with bias in hiring and promotion decisions is to design processes that try to reduce or eliminate the opportunity for bias to affect the choice. The often-cited example is how orchestras changed the auditioning process so that the people choosing new orchestra members wouldn’t be influenced by what the people auditioning look like.
An excellent book about designing fairness into processes is:
What Works
Gender equality by design,
by Iris Bohnet (2016)
Choosing by lottery
One method for reducing the influence of bias in hiring and promotion decisions is to gather a diverse group of people who all meet set requirements to be qualified for the role, and then choose one of those people at random. Sometimes also called sortition.
This idea is not new. People have chosen by lot in many situations, including: academic admissions, immigration, politics, research funding, and more.
Some people want the control of making the choice, believing they know how to choose the best candidate. But research shows that although most people can successfully weed out the unqualified, people are not very good at choosing the best.
Research also shows that choosing leaders by lottery has unanticipated, positive effects beyond a more fair selection. For example, leaders selected by lottery made fewer poor decisions resulting from hubris. (Basically, being told “there were 10 other people equally qualified as you, and you got the job because we drew your name from a hat” dialed down their ego.)
I wanted to see choosing by lottery in action myself…
Example: AI ContentOps squad leader
A few years ago, I created the AI ContentOps squad. The goal of the squad was to give team members a chance to grow their AI skills by working on AI ContentOps projects.
See the following link for an explanation of AI ContentOps:
After the squad had been running for a while, the role of squad leader was created to give team members some leadership experience and to increase their visibility to management.
The squad leader was chosen by lottery:
- All squad members who successfully completed an AI ContentOps project were eligible to be squad leader.
- Eligible squad members were asked if they wanted to be considered for the role.
- The squad leader was drawn from all eligible squad members who confirmed they wanted to be in the running this round.
The role of squad leader was time-limited to 3 months. This made the role less intimidating for people who were trying out a leadership role for the first time. And it meant more people got a chance at the role over time.
Results
Every squad leader did a great job! Each person successfully performed the standard activities of leading the squad. And each person delivered a leadership project of their own choosing that permanently impacted the way the squad functioned — the legacy of their leadership tenure. For every person who took on the role, it was their first time in a formal leadership position within our larger team.
After this method for selecting squad leader had been running for a while, current and former squad members were surveyed about how they felt about the lottery process for choosing squad leaders. Here are some interesting things the survey revealed:
Perception of fairness
- 63% of respondents indicated the usual method for selecting people for roles had sometimes felt unfair in the past.
- But only 12% of respondents thought the lottery selection of squad leader felt unfair.
How it felt to not be selected
- 100% of respondents who were eligible to be squad leader but had not been selected (at the time of the survey) indicated not being selected from the lottery was less disappointing than not being selected by the usual method. In other words, it didn’t feel like a personal rejection when their name just didn’t get drawn.
Impact of selection method on confidence
- 88% of respondents indicated they would have confidence in a leader selected by the lottery method.
- 75% of respondents indicated being selected by the lottery method would not make them feel less confident, compared to the usual selection method.
Less knowledge == less confidence
The first survey question asked respondents if they had been eligible (at the time of the survey) to be squad leader. Answering “I don’t know” to this question was used as a measure of a respondent’s awareness of the selection process.
- People who selected “I don’t know” for that first question indicated they were less confident about the lottery selection process in their answers to the rest of the survey.
- People who selected “I don’t know” for that first question also included mistaken assumptions in their write-in comments for other questions in the survey. For example, suggesting only qualified people should be selected. (Remember, only qualified people can have their name in the draw to begin with. So, by definition, anyone drawn is guaranteed to be qualified.)
Biggest concern: Unqualified people somehow being selected
- Any concerns respondents expressed in survey comments were all centered on the possibility that a lottery method of selecting people for roles might lead to unqualified people being promoted into positions they couldn’t handle. (Of course, that already happens today with the usual selection method described at the beginning of this post. But, setting that aside…) The biggest challenge with implementing a lottery selection method is making sure everyone understands that the only people who can be in the draw are people who are qualified for the role.
Conclusion
To make nontrivial progress in removing the influence of bias in hiring and promotions, it is not enough to do things differently. Instead, we must do different things. Like, using lotteries for selecting people for some roles.
Based on my own experience, here are two requirements for a lottery selection method to be accepted by a team:
- Everyone on the team must understand and accept the sources of bias in existing processes and the impact that bias has.
- Everyone on the team must understand and accept that every candidate whose name is entered into a selection lottery is, by definition, qualified for the role being selected. No unqualified candidates are being put into the lottery, so no unqualified candidates can be selected by the lottery.
Senior team leaders can support lottery selection methods being accepted by teams in two ways:
- Provide education about the ways bias affects existing processes.
- Be supportive of people selected through lottery methods: highlight their credentials, strengths, and experience; and be vocal in support for their decisions and actions. This will help teams be confident about someone selected this way, and this will bolster the confidence of the person selected this way too.
If we needed an adorable puppy, all we’d need to do is find a bunch of adorable puppy candidates, and then select one at random…
